Analysis of Remaining Uncertainties and Exponents Under Various Conditional Rényi Entropies

• Published in: IEEE transactions on information theory 2018-05, Vol.64 (5), p.3734-3755
• Main Authors: Tan, Vincent Y. F., Hayashi, Masahito
• Format: Article
• Language: English
• Subjects: Asymptotic methods; Asymptotic properties; Channel coding; Codes; conditional Rényi entropies; Decay rate; Decoding; Entropy; error exponent; information-theoretic security; Measurement uncertainty; moments of type class enumerator method; Remaining uncertainty; Rényi divergence; Slepian-Wolf coding; Source coding; strong converse exponent; Uncertainty; Uncertainty analysis; universal hash functions
• ISSN: 0018-9448; 1557-9654
• DOI: 10.1109/TIT.2018.2792495

We analyze the asymptotics of the normalized remaining uncertainty of a source when a compressed or hashed version of it and correlated side information is observed. For this system, commonly known as Slepian-Wolf source coding, we establish the optimal (minimum) rate of compression of the source to ensure that the remaining uncertainties vanish. We also study the exponential rate of decay of the remaining uncertainty to zero when the rate is above the optimal rate of compression. In this paper, we consider various classes of random universal hash functions. Instead of measuring remaining uncertainties using traditional Shannon information measures, we do so using two forms of the conditional Rényi entropy. Among other techniques, we employ new one-shot bounds and the moments of type class enumerator method (see Merhav) for these evaluations. We show that these asymptotic results are generalizations of the strong converse exponent and the error exponent of the Slepian-Wolf problem under maximum a posteriori decoding.