Entropy, Optimization and Coding Theory

Entropy is a central concept in many different disciplines, and many important applications involving entropy can be viewed as entropy optimization problems. In this dissertation we study a class of symmetric (conic) programming problems with nonlinear convex objective functions in the context of Euclidean Jordan algebras, which includes the so-called quantum (von Neumann) entropy optimization as a special case. Quantum entropy optimization plays an important role in many areas such as quantum computing, quantum information theory, machine learning, etc. We develop a long-step path-following algorithm for such optimization problems. The theoretical framework is developed for functions compatible (in the sense of Nesterov and Nemirovski) with the standard barrier function - lndet(x). Complexity estimates similar to the case of a linear-quadratic objective function are established, and the theoretical scheme is implemented for the quantum entropy optimization problem. The Shannon entropy optimization problem can be viewed as a special case of the quantum entropy optimization problem with diagonal matrices. We briefly discuss one application of the Shannon entropy optimization in coding theory, namely evaluating the asymptotic pseudoweight enumerators for protograph-based low-density parity-check (LDPC) codes. In the end we also briefly discuss some future work related to quantum relative entropy optimization.