The Generalized Area Theorem and Some of its Consequences

• Published in: IEEE transactions on information theory 2009-11, Vol.55 (11), p.4793-4821
• Main Authors: Measson, C., Montanari, A., Richardson, T.J., Urbanke, R.
• Format: Article
• Language: English
• Subjects: Applied sciences; Area theorem; Back propagation; Belief propagation; belief propagation (BP); Bridges; Channels; Coding, codes; Data transmission; Decoders; Decoding; Entropy; Exact sciences and technology; EXIT curve; Graphs; Information theory; Information, signal and communications theory; Iterative decoding; maximum a posteriori; Maximum likelihood decoding; Maximum likelihood estimation; maximum-likelihood; Maxwell construction; Memoryless systems; Parity check codes; phase transition; Probability; Signal and communications theory; Telecommunications and information theory; Theorems; threshold; Thresholds; Upper bound
• ISSN: 0018-9448; 1557-9654
• DOI: 10.1109/TIT.2009.2030457

There is a fundamental relationship between belief propagation (BP) and maximum a posteriori decoding. The case of transmission over the binary erasure channel was investigated in detail in a companion paper (C. MEacuteasson, A. Montanari, and R. Urbanke, "Maxwell's construction: The hidden bridge between iterative and maximum a posteriori decoding," IEEE Transactions on Information Theory , submitted for publication). This paper investigates the extension to general memoryless channels (paying special attention to the binary case). An area theorem for transmission over general memoryless channels is introduced and some of its many consequences are discussed. We show that this area theorem gives rise to an upper bound on the maximum a posteriori threshold for sparse graph codes. In situations where this bound is tight, the extrinsic soft bit estimates delivered by the BP decoder coincide with the correct a posteriori probabilities above the maximum a posteriori threshold. More generally, it is conjectured that the fundamental relationship between the maximum a posteriori probability (MAP) and the BP decoder which was observed for transmission over the binary erasure channel carries over to the general case. We finally demonstrate that in order for the design rate of an ensemble to approach the capacity under BP decoding the component codes have to be perfectly matched, a statement which is well known for the special case of transmission over the binary erasure channel.