Back in 2004-2005 (in a time I look back fondly on, because I was an OK kid) I was basically a naive techno-optimist about computers and software and AI, but I got seriously worried about Peak Oil.
All the muggles had a "policy level" understanding that the consumer energy economy (and everything in general) would be basically fine, but everyone I could find whose "gears level" understanding of fossil fuel economics was predicting some kind of doom. The futures markets basically said "in 2005, 2009, and 2019 OPEC will politically control the price of oil, and it will be ~$39 per barrel" but that didn't make any object level sense when you dug into the details.
I went kind of crazy, trying to reconcile these things, and read a lot of object level quantitative anthropology trying to figure out whether I was crazy or everyone else was.
What ended up happening is that the economic/technological solution arrived late (but more or less "before serious collapse", like failures of supply chains or the dissolution of traditional constitutions) and also Obama was elected in the midst of a relatively mild "financial collapse" that included oil prices spiking to over $120 per barrel (plus food riots in poor countries).
Since Obama was tribally blue (and the obvious corrective policies were tribally red) and elected with a mandate to solve "the Great Recession" he could get energy extraction reform in a way a red politician could never get away with.
Blue establishment activists objecting to backroom deals like this would be disloyal (only "outsider" ideological leftists, like those involved in the Dakota/Bakken/Standing Rock protests could pragmatically object), and red establishment activist networks were happy to unshackle the frackers and toss a regulatory bone to shale oil. By 2010 things were much less scary, and by 2013 the trajectory of US oil production had totally and dramatically deviated from the predictions inherent to the Hubbert's Peak model of historical oil production.
I consider the 2004-2013 period to have been very personally educational from a "theory of history" perspective :-)
My pet name for the hypothetical field (coined by Michael Flynn in the late 1980's) is "cliology" (named after Clio the Muse of History), and one of many barriers to creating a sociologically viable community of cliology researchers (I'm tempted to call it the "Fundamental Hypothesis of Cliology" as a joke?) is that most major insights in this field are inherently useful for guiding investment and are thus hoarded within the investing class as "one-off trade secrets".
The memetic incentives for serious public knowledge production in this domain would be extremely tricky to set up, and are unlikely to happen except via "great man" or "great circle" interventions. The Fundamental Hypothesis of Cliology suggests that Elon Musk could maybe do it, or a new "thing like the Vienna Circle" might be able to do it, but that's more or less what it would take. Also, even after the initial "boost" from this effort, public research would stall and/or devolve the moment any critical subset of people died, or got day jobs, or got head hunted by a hedge firm, or whatever. The memetic incentive patterns would probably continue to hold for each incremental addition to the field, more or less forever?
So in 2250 (assuming technology keeps advancing and yet there are still autonomous mortal human-shaped minds with their hands on the reins of history) they might very well think that the causality of our period of history was quite retrospectively straightforward... but they will be treating insights that help uniquely predict 2280 (or whatever their window of prediction is) as trade secrets.
You correctly describe these incentives. Assuming both incentives and distribution of information are such, the obvious next research step becomes clear.
A key problem is conveying the reality of such trade secrets and the value of finding them to those without practical or scholarly experience in the area. Only having found such information can one then try to aim to achieve particular outcomes, trying to do so without appropriate knowledge results in wasted or misdirected efforts.
Promoted to the frontpage. While this mentions party politics in an extremely faint way, I would be surprised if this turns into a political discussion, though if it does I will be more careful in the future.
I actually think that history is a particularly underexplored topic for the rationality community, and I am excited to see more writing about history on here. My personal interest lies in the history of the industrial and scientific revolutions, which seem to possibly be the most important things that have happened in all of human history, and with the way they happened pointing at a lot of important facts about the art of rationality – but I haven't really seen that much concretely historical writing on here.
Other references I have found really useful when trying to understand history is the Handbook of Cliometrics (econometric history) and probably my favorite social-science paper in existence: "Why isn't the whole world developed? Lessons from the cotton mills."
Well put.
No one in the world has figured out the true theory of history. If they did, we’d know: they’d be extremely, visibly, powerful.
Hedge fund billionaires like George Soros and Ray Dalio might have the biggest collections of puzzle pieces--they've got the best track records of betting their beliefs.
Some fun quotes from this interview with Dalio (btw, the Financial Times is one of a small list of news publications that hasn't yet given me reasons not to trust it):
Dalio thinks inequality is rising so fast that it has created multiple “economies”: although the elite live in an expanding economy, “for the bottom 60 per cent, 80 per cent, there is a depressed economy that is not growing well”. This means we need to think how we talk about “economics”, he says. America needs a “national commission to rethink our economic metrics”.
But this vision has also changed how he models the future: he thinks this inequality is creating so much strife that it will be political conflict — not economics — that drives markets in 2018 and beyond. “[These days] there’s not the same volatility of inflation, growth and interest rates. So political issues are more important than macro [economic] issues,” Dalio says. “The world was driven by central bank policies [before]. That’s not the case now,” he adds, noting that what investors should watch is not (just) Fed statements, but “the next election in France or in the UK, or how hospitable will Jeremy Corbyn be to capital?”
...
“You can convert whatever you are thinking into an algorithm,” Dalio insists. “We’ve created a conflict gauge looking at words [in the media] and things. We’ve done examinations of all political conflicts in the past and their impact on markets [for models].”
This number-crunching produces some alarming conclusions. Last year, Dalio’s geeks calculated that the proportion of the vote captured by populist candidates had risen from about 7 per cent in 2010 to 35 per cent in 2017. This swing has apparently only ever happened once before, in the 1930s, just before the second world war.
So do the algorithms predict another war? Dalio ducks the question, but admits that he cannot see anything to reverse this trajectory. That is partly because he thinks digital technology is inexorably exacerbating inequality, by eliminating jobs. “We’re headed for a world where you’re either going to be able to write algorithms and speak that language or be replaced by algorithms,” he observes. Another problem is the ever-rising level of global debt. “I am not predicting anything like the type of debt crunch we had in 2008,” he says. “But there is a tightening financial squeeze which is going to hurt the bottom 60 per cent more and more, particularly when we have the next recession.”
Another fun quote:
Dalio believes that some heavily indebted countries, including the United States, will eventually opt for printing money as a way to deal with their debts, which will lead to a collapse in their currency and in their bond markets. “There hasn’t been a case in history where they haven’t eventually printed money and devalued their currency,”
Personally, I suspect that no neat theory of history will ever be found, because human societies are too complicated and idiosyncratic to compress easily. Warren Buffet's partner Charlie Munger stresses the importance of multiple mental models. Phil Tetlock's research suggests that "fox" style thinking beats "hedgehog" style thinking in political judgements. Hedgehogs can be super useful to foxes by building theories for them to use, but in machine learning we see that ensemble models virtually always outperform. I actually think the best way to predict the course of history is to take the problem up a meta level by first becoming an expert on statistics/machine learning. This is a great paper.
I would kind of suspect if people really did take the great man theory seriously, there would be more serious assassination attempts. Maybe they happen more often, and either get caught early or don't get publicized or something.
No one in the world has figured out the true theory of history. If they did, we’d know: they’d be extremely, visibly, powerful.
What if I published the true theory in a book and a billion people read it? Would they all get extremely powerful?
I'm equally skeptical. Suppose you discover it's great man theory and you know you aren't one of those extremely driven, extremely charismatic people?
I expect that, as with most domains of expertise, communicating deep insights rather requires writing at least one textbook (with exercises), and very few people read textbooks. Similar to how you can often understand the argument that a professional mathematician/economist/biologist makes, but they are unable to teach you to be just as good as them.
I think the correct response to the quote would be "Can you not in fact explain some people's great success by their having an accurate implicit theory of history?"
If I take a stab at making my implicit theory explicit (I don’t promise at all this is accurate), I’d say my first-order variables are all very optimisation based. Eukaryotic revolution, the evolution of human brains, creation of de novo artificial intelligence, are all key.
Then for second order variables I’d zoom to human existence and point to key technology innovations - the agricultural and industrial revolutions are top here.
Then for third order variables I’d zoom into small groups of top thinkers in highly selective environments who did either science or institution building, such as Bell Labs, the Royal Society, The Republic of Letters, Manhattan Project, Founding Fathers, and others.
Of course I give that as a loosely defined descriptive theory. I have not stated how one would go about using it to make falsifiable predictions. I notice that the beliefs that ifnrom this theory do heavily inform how I choose to live my life, so I look forward to future posts about how to test and improve my theories.
When it comes to the industrial revolution you can have multiple explanation of why it happened at that specific time.
A common myth is for example that the Greeks had the steam engine but just didn't use it for economic purposes because they had slave. That's wrong. Their copper-wielding ability wasn't good enough to build technology like Thomas Newcomen's steam engine.
Precursor technologies are more important than the common myth acknowledges. Whether it's replacing hand written books with printed books after cheaper paper and durable paper replaced vellum or whether it's targeted phage therapy replacing antibiotics after gene sequencing made it cheap enough to find out what harmful bacteria are in an ill patient, precursor technologies matter a lot.
So. Many. Skulls.
However, theories of history are now back on the menu of serious scholarship. One important name in the field of macrohistory, and the most aligned to the interests of this community in my opinion, is Peter Turchin.
He writes books carefully describing large historical cycles, reviewed by Robin Hanson here.
He is the driving force behind a new interdisciplinary journal, Cliodynamics: The Journal of Quantitative History and Cultural Evolution.
He is also one of the driving forces behind Seshat: Global History Databank.
If anyone wants to explore any or all of these resources as part of an informal study group or similar, I would be interested in hearing from them.
Great example! Talking to him made it clear he was one of the clearer social thinkers around. His project certainly deserves more attention.
No one in the world has figured out the true theory of history. If they did, we’d know: they’d be extremely, visibly, powerful.
No, it might just be that there are no predictable patterns that can be exploited in that way.
No one in the world has figured out the true theory of history. If they did, we’d know: they’d be extremely, visibly, powerful.
This assertion seems to make some implicit assumptions about the true theory of history in itself. Another thing is, "figuring out" is not binary: you can have some insights that take you some of the way, and still there can be a lot of room to do better.
By the way, to the extent that most people care about history at all, they seem to mostly use it to justify the narratives they already believe in rather than actually making predictions.
This is an excerpt from the draft of my upcoming book on great founder theory. It was originally published on SamoBurja.com. You can access the original here.
Why was Barack Obama elected president in 2008? Was it because he ran a smart and successful campaign? Was it because new social media sites allowed young people to get interested in politics? Or was it because American culture was generally shifting away from George W. Bush’s brand of conservatism?
If you read the news articles published on November 4th, 2008, you’ll notice something interesting: journalists explain this historic event in many different ways. Some journalists attribute the campaign’s success primarily to the individual leading the campaign. Others focus more on the influence of new technologies on campaigning. Still others explain it in terms of a general cultural or political shift.
These explanations are revealing—not necessarily of what actually landed President Obama in office, but rather of how each individual journalist conceives of the way things happen in the world. Through their explanations for the outcome of the election, we can glean a bit of their implicit theories of history.
Concept & importance
A theory of history is an explanation of how things generally happen in the world, both in the past and in the future. If, for example, you subscribe to the great man theory of history, then you might explain events by looking at the influential individuals who shaped them. If you subscribe to a technological determinist theory, on the other hand, you might explain events in terms of the technologies that allowed them to occur. Or, you might subscribe to a social determinist theory, explaining both influential individuals and new technologies as the makings of greater societal forces.
Someone operating under the great man paradigm might explain Obama’s election as a product of his and his staff’s exacting efforts in the day-to-day of campaign work. A believer in technological determinism might attribute the win to the unprecedented use of social media, which mobilized previously uninterested voters. Someone adhering to a social determinist view might draw a straight line from the Civil Rights era to the election of Barack Obama, pointing out the inexorable cultural shift towards empowering African-Americans.
Everyone has a theory of history, in that everyone has an explanation of why the world is how it is and an understanding of how the world changes and has changed. Everyone has to: without an understanding of how the world works, no matter how faulty, implicit, or subconscious, we would be prohibited from acting in what we believe is the right way to achieve our goals, whether big or small. Few people could tell you plainly that they are social or technological determinists, or adherents of great man theory. But everyone, if asked, can give reasons why some event or another happened, and whether, or why, it might happen again in the future.
We don’t just explain things with our theories of history. We act on them. If you believe that individuals have the power to significantly shape history, for example, you might be more inclined to make things happen yourself. If you believe that technology drives historical change, you might specifically try to invent new technologies. If, on the other hand, you believe that the fate of the world has already been decided, or if you believe that history is inevitably heading in a certain direction, you may be less inclined to take a stand. After all, if it’s going to happen, then it’s going to happen.
Therefore, whether we’re trying to change the world in a major way or just live our lives in society in the best way possible, it’s vital that we come to understand the true theory of history. We need the true theory of history in order to take the right actions in the world, and we need to accurately predict the results of our actions. If we have an incorrect theory of history, we run the risk of producing unknown and possibly catastrophic consequences, for ourselves or others.
It’s important here to note the distinction between the true theory of history, and the “true” theory of history that we’re aiming for. The true theory of history will be unmanageably complex, because the number of factors that actually influence what happens in the world is incalculably large. Because of its complexity, the true theory of history will be difficult, if not impossible, to use to explain what’s going on in the world. In aiming for the “true” theory of history, we are assuming the power law: we are assuming that there will be a small number of factors that have disproportionately large effects on the world, or that can explain the existence of other factors. We are aiming for a theory that generally explains how things happen in the world. Going forward, we will drop the quotation marks and stipulate that the true theory of history is the theory that takes into account the core causes contributing to the world as it exists, making it comprehensible and usable to us mere mortals.
No one has it
No one in the world has figured out the true theory of history. If they did, we’d know: not only would they be extremely, visibly, powerful, but they would be active in many domains—politics, religion, culture, technology—reshaping society step by step, or taking seemingly prescient advantage of trends, with many successes and few false starts. There are historical examples of incredible individuals, such as the Indian Emperor Ashoka the Great, and organizations, such as the Catholic Church, whose repeated success across multiple domains is difficult to explain without them understanding at least fragments of a true theory of history.
There are many reasons why no one has figured out the full true theory of history, some psychological and some practical.
There are at least three psychological reasons for why most people are deterred from finding the true theory of history. The first is that the vast majority of people only have an implicit, or subconscious, theory of history. In other words, most people do not even have the concept of a theory of history. The problem with relying on your implicit theory of history is that it’s wrong, without a doubt. The world is complex, and your theory of history has to explain how everything in the world works. Without explicitly trying to improve your theory of history, there is no hope: there will be countless things that you have not had the time or perspective to take into account. Improving your theory of history implicitly is not systematic enough to work.
The second reason why no one has managed to achieve the true theory of history is that many people endorse one theory of history while unknowingly acting on another. For example, some people explicitly endorse the technological determinist view of history even as they implicitly act on the great man paradigm: believing that it will require the work of remarkable individuals to create the technology that will save the world, for example, instead of believing that the inevitable, impersonal progress of technology will do so.
There can be many belief-based reasons for why people fall into this trap, but on a more basic level, people simply don’t have a good sense of what their implicit theories of history are, or know how to explicate them, which means they cannot reliably align their intellectual and emotional beliefs. To some extent, acting on your implicit theory of history while operating under a different explicit theory is fine — after all, your implicit theory will, for a while, be more nuanced than your explicit one. What is problematic is to act unconsciously on one theory of history and proclaim another; this makes it very difficult to improve your implicit theory of history, which you act on.
The third reason is that people tend to switch between theories of history in an unprincipled way, which prevents them from noticing theory-threatening anomalies. If they can’t notice and explain seeming anomalies in their theory of history, then they can’t improve their theory. If someone largely adheres to the great man paradigm, for example, but resolves any contradictions by falling back on the technological determinist view, then they’ve prevented themselves from justifying their understanding of the great man theory, or realizing that their justification is inadequate or incorrect. Theory-threatening anomalies have to be resolved, not rationalized.
These are just a few of the psychological barriers that prevent people from making progress towards the true theory of history. But there’s a simpler, more practical problem: the world is complex. In order to understand it, you need the right methodology, and you need a huge amount of properly processed data.
Read more from Samo Burja here.