Information Theory

Information theory is the mathematical treatment of the concepts, parameters and rules governing the transmission of messages through communication systems. The techniques used in information theory are probabilistic in nature and some view information theory as a branch of probability theory. Information theory also provides methodologies to separate real information from noise and to determine the channel capacity required for optimal transmission conditioned on the transmission rate.