4 postulates of information theory
1. Information is defined in the context of a set of alternatives. The amount of information quantifies what is necessary to specify an outcome from the alternatives
2. The amount of information is independent of the semantics
3. Information is always positive
4. Information is measured on a logarithmic scale
? Probabilities are multiplicative, but we want information to be additive