Information theory

Some of the key information-theoretic measures of uncertainty and statistical dependence, such a entropy and mutual information are introduced, such a entropy and mutual information.