Information Theory of Startups

The Hard Ramp

Information Theory

Around 1 year ago today, I was reading The Art of Doing Science and Engineering, by Richard Hamming, a UIUC and Bell Labs alum. The book dives into a concept pioneered by engineer Claude Shannon called Information Theory. After I read it, I couldn’t help but think that this framework applies to nearly every aspect of human life.

The basic intuition behind information theory is that unlikely information is much more valuable than likely information. A very simple concept, yet so much of society is built around doing likely things. There are two separate frameworks for this tug-of-war. We will discuss both.

Likely Things

Most things in our world are likely things. When you go through high school, you are participating in a structured curriculum that all those around you are. When you attend college, you are given more freedom, but ultimately you are navigating existing dogma. When you join a company, you are joining an ongoing concern. However, there was a point in recent history where each of these curriculums and processes were mapped out on a white sheet of paper for the first time. 

For the most part, likely things are extremely valuable. Especially if they have remained likely things for long periods of time. For instance, financial institutions, restaurants, and cemeteries have been around for a very long time. There is consistent value in them, and thus they persist.

Unlikely Things

There is value in doing unlikely things that are both novel and valuable. Creating new things. Combining existing things in novel ways. Creating a novel signal that prunes (disrupts) existing systems. Etc. Unlikely things that are valuable are equivalent to going from 0-1.

In this sense, inventions are underappreciated. It’s not easily apparent just how much work went into everything around us. The chair you sit on, the pen you’re writing with, and every single component that makes up the iPhone. These seem like small creations, but now our world is built upon their foundation. 

The counter-argument to this is that it doesn’t matter much who invented things initially, problems are apparent themselves, and it is clear someone will capitalize on them, but it requires the skills, intelligence, and institutional backing to do so. Going 1-n could very well be more important. 

1-n

A passage from Pieces of the Action, by Vannevar Bush.

What is more valuable? The initial invention or the following work done to bring something into reality?

I don’t know, and the answer is probably complicated and contextually dependent. However, I am personally not interested in doing things that others can do, and frankly, I don’t think I’m that good at them. It’s probably a different skill set. You kind of get hit with that reality in investment banking. You work hard and excel for a decade to get a good job, then you find there are others who are still much better than you. Further, the best analysts are themselves commodities. You almost can’t win. Even if you do win and “make it,” what is it you’ve actually won?

From a young age, I remember not having any interest in doing things that other people could do or were better than me at. It may be the logical and rational decision to do so, but not all of my decisions are made with logic and reason. In high school, my guiding quote was Robert Frost’s, and it seems to still be.

“Two roads diverged in a wood, and I—

I took the one less traveled by,

And that has made all the difference.

Time Changes

In the dark ages, there probably wasn’t too much value in doing new things. There may have even been too much innovation at certain points in history and not enough people commercializing them.

Today and in the future, the surface area of the fractal of new technology grows at an unprecedented pace. We should plan for a large number of previously unsolvable problems to present themselves as new opportunities in the years to come. Since humans don’t understand exponential growth well, my bet is that we will be perpetually lagging in novel, unlikely, and valuable things for the decades to come.

As a principled belief, I generally consider the bottleneck of progress to be the people who actually build the damn thing or have the agency to work incredibly hard / focus on solving problems. The initial ramp is the hardest.

Of Commodities and Men

The question that I ask myself is what value I provide. I generally consider myself a stupid and unhelpful person of few talents. However, if I connect the dots backwards, I can recognize a pattern of environments where I have done well connecting people, ideas, and capital in new ways. I also really like doing it.

While being a Tyro is unhelpful in most scenarios, I think there is value in being young & dumb. You see the world with fresh eyes. I like questioning existing systems.

There is a Need for More Unlikely Things

There is no value in doing new things for the sake of doing new things, but it is my observation that the vast majority of people in the world are not doing new things. And some of the people I saw doing new things in school and SF were seemingly doing new things because they like the idea of doing new things. 

“At a time when the growth of useful knowledge is expanding exponentially, which leads to increased opportunities for growth, society has bred citizens who aren’t irreverent or hard working enough to pursue them.” - yes, i'm quoting myself

Even in silicon valley, there aren’t enough people doing that much new stuff. Humans are the most sheep-like, domesticated animal of them all. There are more problems that need to be solved than people to solve them at this moment. Many of the AI companies are solving the same exact problem.

The observation I also had from UIUC is that many students were chomping at the bit to join exciting “new” things, but most people don’t themselves start them. If that is the case, it feels like my moral obligation to tackle the initial curve of hard problems. Vannevar Bush highlights the need for far-seeing and energetic individuals in advancing technological progress.

Vision

Richard Hamming remarks two quotes in the book that effectively say the same thing on the topic of “your research”. I believe having a clear, novel, useful, and determinate vision is of the utmost importance in my life.

“Having a vision is what separates the leaders and the followers.”

“It is well known the drunken sailor who staggers to the left or right with n independent random steps will, on the average, end up about √n steps from the origin. But if there is a pretty girl in one direction, then his steps will tend to go in that direction and he will go a distance proportional to n. In a lifetime of many, many independent choices, small and large, a career with a vision will get you a distance proportional to n, while no vision will get you only the distance √n. In a sense, the main difference between those who go far and those who do not is some people have a vision and the others do not and therefore can only react to the current events as they happen.”

(a somewhat crude example but I’m with it)

Conclusion

—Where frameworks diverge—

Doing likely and unlikely things contribute different forms of value to society. I simply believe that my nature is that of someone who does unlikely things.

The counter-argument I have is that at nearly every instance in my life where I have itched to depart the trodden path (not go to college, not start a job, etc), there has been a single person to suggest I didn’t know all I thought I did. While there is no way to run the experiment again, they were probably right. At some point, however, you need to change the independent variable and see what happens.

More importantly, there is an instinctual aspect of human nature where men are supposed to wander into the unknown, burn the boats, work on important problems that no one else is, accept failure as the likely outcome, yet strive for excellence regardless. Failure guides, teaches, and sharpens us. We are supposed to do the hard thing exactly because it is hard.