|
|
> Try adapting Shannon's information theory. If we assume a probability
> distribution on the set of messages, then the information of a given
> message m is given by -log(p(m)) [1]. That is, the less likely a
> message, the more informative.
Since only frequencies of symbols are counted, entropy depends only on
them.
A set of random numbers can have the form
0,0,0,0...1,1,1,1...2,2,2,2...
It has the same entropy as any number (permutation) formed from this
set.
The strings of random numbers are randomly ordered digits.
Any sensefull text is a carefull ordered string of symbols.
Newertheless, the long distances between consecutive symbols (or words,
if we take them as the smallest information unit) have properties of
randomly ordered strings.
The number e is generated by an algorithm. Newertheless, the distances
between its numerals are quite random.
kunzmilan
|
|