Shared Frames Are Coordination Tools

When Claude Shannon invented information theory, he was working at Bell Labs. Naturally, the new theory was applied to Bell’s telephone system quite quickly. But information theory itself did not offer any immediately-usable methods to run more phone calls through the same wires; practical compression algorithms came later. So how did information theory create immediate practical value?

What information theory offered was a new way of framing communication infrastructure problems. Historically, the content transmitted (e.g. speech) was tightly coupled to the mode of transmission (e.g. voltage oscillations matching the audio oscillations). Information theory suggested that the content and the mode of transmission could be decoupled. Any kind of data (written words, speech, video) could in-principle all be transmitted over any kind of channel (copper wire, radio, microwave, messenger pigeon) with near-optimal efficiency and arbitrarily-low noise. We could even quantify “amount of information” in a message and “information capacity” of a channel, independently of each other, in terms of “bits” - a universal way to represent information.

So information theory naturally suggests a way to break up the problem of communication infrastructure design: separate content and encoding from transmission and channel design. More generally, information theory is a frame: it suggests questions, approximations, what to pay attention to vs ignore, and ways to break up the problem/system into subproblems/subsystems.

Questions:
- What’s the information capacity of a given channel (e.g. a pair of microwave towers), and how can we increase it?
- What’s the amount of information per second in a phone call, and how can we efficiently encode it?
Approximations:
- Accurately transmitting an n-bit message through a channel requires a channel capacity of approximately n bits, regardless of the message details.
- Conversely, an n-bit capacity channel can accurately transmit a message of approximately n bits, regardless of the channel details.
What to pay attention to vs ignore:
- Ignore most interactions between channel-design and message content - e.g. we don’t need to worry about the idiosyncrasies of human speech when designing a satellite link.
- Pay attention to how many bits per second we can send through the channel (i.e. throughput).
- Pay attention to how long th

...

csqr

csqr