Some excerpts from Peter Thiel's 2014 book Zero to One that I've repeatedly come back to over the years:

[...] Why has so much of our society come to believe that there are no hard secrets left? It might start with geography. There are no blank spaces left on the map anymore. If you grew up in the 18th century, there were still new places to go. After hearing tales of foreign adventure, you could become an explorer yourself. This was probably true up through the 19th and early 20th centuries; after that point photography from National Geographic showed every Westerner what even the most exotic, underexplored places on earth look like. Today, explorers are found mostly in history books and children’s tales. Parents don’t expect their kids to become explorers any more than they expect them to become pirates or sultans. Perhaps there are a few dozen uncontacted tribes somewhere deep in the Amazon, and we know there remains one last earthly frontier in the depths of the oceans. But the unknown seems less accessible than ever.

Along with the natural fact that physical frontiers have receded, four social trends have conspired to root out belief in secrets. First is incrementalism. From an early age, we are taught that the right way to do things is to proceed one very small step at a time, day by day, grade by grade. If you overachieve and end up learning something that’s not on the test, you won’t receive credit for it. But in exchange for doing exactly what’s asked of you (and for doing it just a bit better than your peers), you’ll get an A. This process extends all the way up through the tenure track, which is why academics usually chase large numbers of trivial publications instead of new frontiers.

Second is risk aversion. People are scared of secrets because they are scared of being wrong. By definition, a secret hasn’t been vetted by the mainstream. If your goal is to never make a mistake in your life, you shouldn’t look for secrets. The prospect of being lonely but right—dedicating your life to something that no one else believes in—is already hard. The prospect of being lonely and wrong can be unbearable.

Third is complacency. Social elites have the most freedom and ability to explore new thinking, but they seem to believe in secrets the least. Why search for a new secret if you can comfortably collect rents on everything that has already been done? Every fall, the deans at top law schools and business schools welcome the incoming class with the same implicit message: "You got into this elite institution. Your worries are over. You’re set for life." But that’s probably the kind of thing that’s true only if you don’t believe it.

Fourth is "flatness." As globalization advances, people perceive the world as one homogeneous, highly competitive marketplace: the world is "flat." Given that assumption, anyone who might have had the ambition to look for a secret will first ask himself: if it were possible to discover something new, wouldn’t someone from the faceless global talent pool of smarter and more creative people have found it already? This voice of doubt can dissuade people from even starting to look for secrets in a world that seems too big a place for any individual to contribute something unique.

There’s an optimistic way to describe the result of these trends: today, you can’t start a cult. Forty years ago, people were more open to the idea that not all knowledge was widely known. From the Communist Party to the Hare Krishnas, large numbers of people thought they could join some enlightened vanguard that would show them the Way. Very few people take unorthodox ideas seriously today, and the mainstream sees that as a sign of progress. We can be glad that there are fewer crazy cults now, yet that gain has come at great cost: we have given up our sense of wonder at secrets left to be discovered.

 

The World According to Convention

How must you see the world if you don’t believe in secrets? You’d have to believe we’ve already solved all great questions. If today’s conventions are correct, we can afford to be smug and complacent: "God’s in His heaven, All’s right with the world."

For example, a world without secrets would enjoy a perfect understanding of justice. Every injustice necessarily involves a moral truth that very few people recognize early on: in a democratic society, a wrongful practice persists only when most people don’t perceive it to be unjust. At first, only a small minority of abolitionists knew that slavery was evil; that view has rightly become conventional, but it was still a secret in the early 19th century. To say that there are no secrets left today would mean that we live in a society with no hidden injustices.

In economics, disbelief in secrets leads to faith in efficient markets. But the existence of financial bubbles shows that markets can have extraordinary inefficiencies. (And the more people believe in efficiency, the bigger the bubbles get.) In 1999, nobody wanted to believe that the internet was irrationally overvalued. The same was true of housing in 2005: Fed chairman Alan Greenspan had to acknowledge some "signs of froth in local markets" but stated that "a bubble in home prices for the nation as a whole does not appear likely." The market reflected all knowable information and couldn’t be questioned. Then home prices fell across the country, and the financial crisis of 2008 wiped out trillions. The future turned out to hold many secrets that economists could not make vanish simply by ignoring them. [...]

Thiel argues that there are many discoverable (or discovered-but-not-widely-known) truths you can use to get an edge, make plans, and deliberately engineer a better future.

Some general social facts Thiel cites to argue that people in the US are less interested in secrets than they were in, e.g., the 1950s:

• From ch. 1: We live in an age of globalization, rather than technology innovation—outside of information technology, we've seen a Great Stagnation in new ideas and innovations since the 1970s-80s. We speak as though prosperous nations are "developed" as opposed to "developing," and focus on improving the world by spreading ideas that have already worked, rather than by coming up with radically new ideas.
• From ch. 2: As a result of the Dot-Com Bubble, even IT has become allergic to developing big new plans and ideas.
• From ch. 6: The US in the 1950s had "definite optimism," whereas the present-day US has "indefinite optimism". We think things are going to get better, but we're skeptical that we can learn anything that will help us concretely plan or invent the better thing.
• From ch. 13: The 2005-2010 cleantech bubble again shows companies relying on incremental improvements, conventional and widely shared knowledge, and vague ungrounded optimism.

In an indefinite world, according to Thiel...

Process trumps substance: when people lack concrete plans to carry out, they use formal rules to assemble a portfolio of various options. This describes Americans today. In middle school, we’re encouraged to start hoarding "extracurricular activities." In high school, ambitious students compete even harder to appear omnicompetent. By the time a student gets to college, he’s spent a decade curating a bewilderingly diverse résumé to prepare for a completely unknowable future. Come what may, he’s ready—for nothing in particular. [...]

Instead of working for years to build a new product, indefinite optimists rearrange already-invented ones. Bankers make money by rearranging the capital structures of already existing companies. Lawyers resolve disputes over old things or help other people structure their affairs. And private equity investors and management consultants don’t start new businesses; they squeeze extra efficiency from old ones with incessant procedural optimizations. It’s no surprise that these fields all attract disproportionate numbers of high-achieving Ivy League optionality chasers; what could be a more appropriate reward for two decades of résumé-building than a seemingly elite, process-oriented career that promises to "keep options open"? [...]

While a definitely optimistic future would need engineers to design underwater cities and settlements in space, an indefinitely optimistic future calls for more bankers and lawyers. Finance epitomizes indefinite thinking because it’s the only way to make money when you have no idea how to create wealth. If they don’t go to law school, bright college graduates head to Wall Street precisely because they have no real plan for their careers. And once they arrive at Goldman, they find that even inside finance, everything is indefinite. It’s still optimistic—you wouldn’t play in the markets if you expected to lose—but the fundamental tenet is that the market is random; you can’t know anything specific or substantive; diversification becomes supremely important.

 

Indefinite Finance

The indefiniteness of finance can be bizarre. Think about what happens when successful entrepreneurs sell their company. What do they do with the money? In a financialized world, it unfolds like this:

• The founders don’t know what to do with it, so they give it to a large bank.
• The bankers don’t know what to do with it, so they diversify by spreading it across a portfolio of institutional investors.
• Institutional investors don’t know what to do with their managed capital, so they diversify by amassing a portfolio of stocks.
• Companies try to increase their share price by generating free cash flows. If they do, they issue dividends or buy back shares and the cycle repeats.

At no point does anyone in the chain know what to do with money in the real economy. But in an indefinite world, people actually prefer unlimited optionality; money is more valuable than anything you could possibly do with it. Only in a definite future is money a means to an end, not the end itself.

 

Indefinite Politics

Politicians have always been officially accountable to the public at election time, but today they are attuned to what the public thinks at every moment. Modern polling enables politicians to tailor their image to match preexisting public opinion exactly, so for the most part, they do. Nate Silver’s election predictions are remarkably accurate, but even more remarkable is how big a story they become every four years. We are more fascinated today by statistical predictions of what the country will be thinking in a few weeks’ time than by visionary predictions of what the country will look like 10 or 20 years from now.

And it’s not just the electoral process—the very character of government has become indefinite, too. The government used to be able to coordinate complex solutions to problems like atomic weaponry and lunar exploration. But today, after 40 years of indefinite creep, the government mainly just provides insurance; our solutions to big problems are Medicare, Social Security, and a dizzying array of other transfer payment programs. It’s no surprise that entitlement spending has eclipsed discretionary spending every year since 1975. To increase discretionary spending we’d need definite plans to solve specific problems. But according to the indefinite logic of entitlement spending, we can make things better just by sending out more checks. [...]

 

Indefinite Philosophy

From Herbert Spencer on the right and Hegel in the center to Marx on the left, the 19th century shared a belief in progress. (Remember Marx and Engels’s encomium to the technological triumphs of capitalism from this page.) These thinkers expected material advances to fundamentally change human life for the better: they were definite optimists.

In the late 20th century, indefinite philosophies came to the fore. The two dominant political thinkers, John Rawls and Robert Nozick, are usually seen as stark opposites: on the egalitarian left, Rawls was concerned with questions of fairness and distribution; on the libertarian right, Nozick focused on maximizing individual freedom. They both believed that people could get along with each other peacefully, so unlike the ancients, they were optimistic. But unlike Spencer or Marx, Rawls and Nozick were indefinite optimists: they didn’t have any specific vision of the future. [...]

Today, we exaggerate the differences between left-liberal egalitarianism and libertarian individualism because almost everyone shares their common indefinite attitude. In philosophy, politics, and business, too, arguing over process has become a way to endlessly defer making concrete plans for a better future.

 

Indefinite Life

Our ancestors sought to understand and extend the human lifespan. In the 16th century, conquistadors searched the jungles of Florida for a Fountain of Youth. Francis Bacon wrote that “the prolongation of life” should be considered its own branch of medicine—and the noblest. In the 1660s, Robert Boyle placed life extension (along with "the Recovery of Youth") atop his famous wish list for the future of science. Whether through geographic exploration or laboratory research, the best minds of the Renaissance thought of death as something to defeat. (Some resisters were killed in action: Bacon caught pneumonia and died in 1626 while experimenting to see if he could extend a chicken’s life by freezing it in the snow.)

We haven’t yet uncovered the secrets of life, but insurers and statisticians in the 19th century successfully revealed a secret about death that still governs our thinking today: they discovered how to reduce it to a mathematical probability. "Life tables" tell us our chances of dying in any given year, something previous generations didn’t know. However, in exchange for better insurance contracts, we seem to have given up the search for secrets about longevity. Systematic knowledge of the current range of human lifespans has made that range seem natural. Today our society is permeated by the twin ideas that death is both inevitable and random.

Meanwhile, probabilistic attitudes have come to shape the agenda of biology itself. In 1928, Scottish scientist Alexander Fleming found that a mysterious antibacterial fungus had grown on a petri dish he’d forgotten to cover in his laboratory: he discovered penicillin by accident. Scientists have sought to harness the power of chance ever since. Modern drug discovery aims to amplify Fleming’s serendipitous circumstances a millionfold: pharmaceutical companies search through combinations of molecular compounds at random, hoping to find a hit.

But it’s not working as well as it used to. Despite dramatic advances over the past two centuries, in recent decades biotechnology hasn’t met the expectations of investors—or patients. Eroom’s law—that’s Moore’s law backward—observes that the number of new drugs approved per billion dollars spent on R&D has halved every nine years since 1950. [...]

Biotech startups are an extreme example of indefinite thinking. Researchers experiment with things that just might work instead of refining definite theories about how the body’s systems operate. Biologists say they need to work this way because the underlying biology is hard. According to them, IT startups work because we created computers ourselves and designed them to reliably obey our commands. Biotech is difficult because we didn’t design our bodies, and the more we learn about them, the more complex they turn out to be.

But today it’s possible to wonder whether the genuine difficulty of biology has become an excuse for biotech startups’ indefinite approach to business in general. Most of the people involved expect some things to work eventually, but few want to commit to a specific company with the level of intensity necessary for success. It starts with the professors who often become part-time consultants instead of full-time employees—even for the biotech startups that begin from their own research. Then everyone else imitates the professors’ indefinite attitude.

On Thiel's account, people don't believe in secrets, but they do believe in mysteries or things we can't figure out today, though we might know them at some point in the indefinite future. The indefinite optimism he's criticizing doesn't assume we're omniscient, but it assumes that there are relatively few cheat codes or exploits an individual can discover, especially in the domain of "altering and predicting the long-term future".

New discoveries spontaneously pop out of a slot machine, and then go straight to the textbook or the trash heap; and only the gullible will favor unpopular ideas over popular ones.