Footnote: Algorithmic information
Computers represent information as stored symbols
- Not probabilistic (in the Shannon sense)
- Can we quantify information from an algorithmic standpoint?
Kolmogorov complexity: Algorithmic randomness of a string
- Alternative definition of information content
- Based on the theory of computation rather than on probabilities
- The length of the shortest program p (in bits) that, when run on a UTM, produces the string s
The Kolmogorov complexity is an uncomputable function
- Similar argument to the halting problem
- How do we know when we have the shortest program?