An error correcting code (ECC) is a way of controlling errors in data that is being transmitted over an unreliable or noisy communication channel. In an ECC, the sender encodes the message with redundant information. The receiver is able to detect a limited number of errors, and to correct these errors without retransmission.
Turbo codes are a class of ECC that approach very closely the theoretical optimum for an ECC. Turbo codes are used in mobile and satellite communications.
Turbo codes can be understood as an example of loopy belief propagation in Bayesian networks. The famous 1998 paper that first pointed out this fact is
“Turbo decoding as an instance of Pearl’s “belief propagation algor…, by R. McEliece, D. MacKay & J. Cheng.
R. McEliece was a very famous information theorist from Caltech.
D. MacKay was a super-nice guy. I actually wrote an email to him once, and he very graciously replied. Alas, he died in 2016 at age 48. He is sorely missed by his numerous admirers like me. MacKay wrote an excellent information theory book, Information Theory, Inference, and Learning Algorithms, which is available for free in pdf form, or for a modest fee in paper form.
To learn classical info theory, I recommend D. MacKay’s book and the canonical text by Cover & Thomas. While reading those books, it becomes painfully obvious to any B net aficionado that classical info theory can be easily visualized with B nets, and that it is very helpful to do so. To see what I mean, see, for instance,
Shannon Information Theory Without Shedding Tears Over Delta &… by Robert R. Tucci
Oh, and one more thing…
Much of quantum information theory can be visualized in terms of quantum B nets. See, for example,
Quantum Information Theory – A Quantum Bayesian Net Perspective, by Robert R. Tucci
This post was brought to you by www.ar-tiste.xyz, purveyors of high quality and low cost Bayesian Network Software & Services
Credit: Data Science Central By: Robert R. Tucci
