Information Theory – Ain’t Nobody Got Time For That!

Information Theory“Information Theory” is the short name given to Claude Shannon’s Mathematical Theory of Communication, a 1948 paper that laid the groundwork for the information age.It is not as much about information as it is about the transmission of information. That is, communication.The theory has had enormous implications on modern society, and there are still limitless applications to ponder and pursue. For now though, I’ll simply explain three of the main breakthroughs and hope you find them as cool-in-themselves as I do:The components of communicationHow to communicate a greater quantity at a faster speedHow to transcend noise and communicate accurately1. A Unified Framework of CommunicationThe first part of Information Theory says that all communication — any message across any medium — fits into the following framework:The message starts from a source, such as a human or a machine. It gets expressed via a transmitter, such as a letter or a tweet or your vocal chords. It then travels over a channel, like airwaves, where it is subject to noise. It hits a receiver, like someone’s mailbox or twitter feed. Lastly, it arrives (or maybe doesn’t) at a destination, such as someone’s brain. Landlines are easy to visualize. Phones act as the transmitter and receiver, and the cable is the channel.Next, Shannon posited that in addition to a common framework for communication, there is also a common thing that is transmitted when you communicate. He called this thing “information.”According to Shannon’s definition, something contains information if it tells you something new. Its units are measured in “binary digits” (0 or 1), better known by the condensed form “bits.”A coin toss makes for the simplest example. If I flip a fair coin, the result can be either heads or tails (0 or 1), so the outcome of the coin toss contains one bit of information. A trick coin that has heads on both sides, on the other hand, contains no information because it can’t possibly tell me anything new.* No new informationA message (or piece of a message) that doesn’t tell you anything new is called redundancy. For example, when someone says “every single one…,” the word “single” is redundant. You can drop it and still retain the same information. Shannon estimated that the English language is 50% redundancy.2. How to Communicate More, FasterBeing able to quantify information is significant because it allows us to compress messages by squeezing out redundancy.Continuing with the example of the English language, if you know that “u” always follows the letter “q”, then you can drop the “u” before it enters the channel and add it back in when it gets to the other side.QUITE A LOVELY DAY! -> QITE LOVLY DAY!-> QUITE A LOVELY DAY!This is what led to the Information Age — communicating more and more with fewer and fewer bits.2. Maintaining AccuracyHowever, as with any time one tries to increase efficiency, there is a balance in the form of quality and effectiveness. If I strip out all the redundancy in my words when I speak to you, it’s going to be harder to catch everything and the likelihood that you misunderstand the message will increase.Further, we must assume some level of noise/interference in the communication channel. Noise can come in many forms. In the case of a phone call, maybe it’s physical background noise, or distractions the person on the other line faces.So how do we combat noise and mitigate the risk of communication failure? By adding redundancy back in!Indeed, this is why languages have redundancy in the first place!Ain’t Nobody Got Time for MathShannon is often compared to Einstein, and I will draw an analogy here as well.Information Theory, like E = mc² , was earth-shattering mainly because the math checked out and allowed engineers to put the concepts into practice. If Einstein just said the concept that energy and mass are two sides of the same coin, it would not have led to a Nobel prize.But it is the concepts that I find so interesting, and that allow us to extend the theory to other disciplines and applications.I am looking to apply Information Theory to self-improvement/self-actualization by conceptualizing communication from your higher self to your current self.** I know… it’s heady. I’m currently writing a book to fully expand on this, but a few thoughts to start:We can leverage Shannon’s communication framework by making sure that all the pieces are in place. In particular, we need to work on the receiving end (when you realize insights, where do they go?). Write things down!We can use tracking to compress experiences and feelings. (dashboard example)We can add in redundancy by reviewing our goals, tasks, insights, and data in a systematic way.Please share your thoughts in the comments and consider supporting me on Patreon!*Note that information in Shannon’s theory does NOT equal meaning. Jibberish nonsense, for example, IS information because the person hearing it would not have known it before. Same with a fair coin toss.**We can apply the same concept to organizations too, but they are more complex to think about.
SIGUE LEYENDO…
Ir a la fuente / Author: Chelsea Ursaner