Claude Shannon
1916–2001Mathematician and electrical engineer who founded information theory — the foundation for modern communication, compression, and machine learning. His 1948 paper on entropy and channel capacity quietly enabled the entire digital age, from JPEG to 5G to LLM tokenisers.
- A Mathematical Theory of Communication (1948) — bits, entropy, channel capacity.
- Boolean logic in circuits (1937 master's thesis) — the basis for digital computers.
- Early chess-playing programs.
FAQ
What is information theory?
The mathematical theory of communication Shannon founded in his 1948 paper. It defines bits as the unit of information, quantifies entropy as a measure of uncertainty, and proves how much data a channel can carry — concepts every codec, file format, and language model still rests on.
What was Shannon's master's thesis about?
His 1937 MIT thesis showed that Boolean algebra could analyse and design electrical switching circuits. It established the link between symbolic logic and physical hardware — the basis for every digital computer that followed.
What is the Shannon-Hartley theorem?
A formula that sets the maximum reliable data rate of a noisy communications channel given its bandwidth and signal-to-noise ratio. It is the theoretical ceiling that every modem, Wi-Fi standard, and 5G cell tower designer plans against.
Why does Shannon matter to modern AI?
Modern AI runs on Shannon's primitives: cross-entropy loss is information-theoretic, tokenisers minimise expected bit-length, and the lossless-compression view of large language models is now a respected framing of what they are doing.