Runtime Arguments
38: The Claude Shannon Episode: Surprise, Entropy, and Bits
1:03:37
What is Information Theory?
Information is not meaning
Likely things need very few bits to communicate, maybe less than 1! Unlikely things need more bits.
Entropy is in the eye of the beholder. It’s what you already know that informs your ability to extract meaning from a message. To one person, it’s just noise; to one who holds the encryption key, there may be way less information’s, but m…