Duality between source coding and channel coding and its extension to the side information case

We explore the information-theoretic duality between source coding with side information at the decoder and channel coding with side information at the encoder. We begin with a mathematical characterization of the functional duality between classical source and channel coding, formulating the precis...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on information theory 2003-05, Vol.49 (5), p.1181-1203
Main Authors: Pradhan, S.S., Chou, J., Ramchandran, K.
Format: Article
Language:English
Subjects:
Channel coding
Codes
Computer aided software engineering
Costs
Decoding
Distortion measurement
Information systems
Information theory
Mutual information
Quantization
Sensor systems and applications
Source coding
Theory
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We explore the information-theoretic duality between source coding with side information at the decoder and channel coding with side information at the encoder. We begin with a mathematical characterization of the functional duality between classical source and channel coding, formulating the precise conditions under which the optimal encoder for one problem is functionally identical to the optimal decoder for the other problem. We then extend this functional duality to the case of coding with side information. By invoking this duality, we are able to generalize the result of Wyner and Ziv (1976) relating to no rate loss for source coding with side information from Gaussian to more arbitrary distributions. We consider several examples corresponding to both discrete- and continuous-valued cases to illustrate our formulation. For the Gaussian cases of coding with side information, we invoke geometric arguments to provide further insights into their duality. Our geometric treatment inspires the construction and dual use of practical coset codes for a large class of emerging applications for coding with side information, such as distributed sensor networks, watermarking, and information-hiding communication systems.
ISSN:0018-9448
1557-9654
DOI:10.1109/TIT.2003.810622