Entropy Calculator
In information theory, entropy is a measure of the uncertainty associated
with a random variable. In this context, the term usually refers to the
Shannon entropy, which quantifies the expected value of the information
% contained in a message, usually in units such as bits. In this context, a
'message' means a specific realization of the random variable.
Shannon denoted the entropy H of a discrete random variable X with possible
values {x1, ..., xn} as,
H(X) = E(I(X)).
E is the expected value,
I is the information content of X
Example usage
-------------
in = [.25 .25 .25 .25];
b = 'bit';
Entropy = info_entropy (in, b)