Cross-posting from my personal blog, but written primarily for Less Wrong after recent discussion here.
There are whispers that the Efficient-Market Hypothesis is dead. Eliezer's faith has been shaken. Scott says EMH may have been the real victim of the coronavirus.
The EMH states that “asset prices reflect all available information”. The direct implication is that if you don’t have any non-available information, you shouldn’t expect to be able to beat the market, except by chance.
But some people were able to preempt the corona crash, without any special knowledge! Jacob mentioned selling some of his stocks before the market reacted. Wei Dai bought out-of-the-money 'put' options, and took a very handsome profit. Others shorted the market.
These people were reading the same news and reports as everyone else. They profited on the basis of public information that should have been priced in.
And so, the EMH is dead, or dying, or at the very least, has a very nasty-sounding cough.
I think that rumours of the death of efficient markets have been greatly exaggerated. It seems to me the EMH is very much alive-and-kicking, and the recent discussion often involves common misunderstandings that it might be helpful to iron out.
This necessarily involves pissing on people's parade, which is not much fun. So it's important to say upfront that although I don't know Wei Dai, he is no doubt a brilliant guy, that Jacob is my favourite blogger in the diaspora, that I would give my left testicle to have Scott's writing talent and ridiculous work ethic, that Eliezer is a legend whose work I have personally benefited from greatly, etc.
But in the spirit of the whole rationality thing, I want to gently challenge what looks more like a case of 'back-slaps for the boys' than a death knell for efficient markets.
First: how the heck did the market get the coronavirus so wrong?
The Great Coronavirus Trade
Lots of people initially underreacted to COVID-19. We are only human. But the stockmarket is not only human—it’s meant to be better than this.
Here’s Scott, in A Failure, But Not of Prediction:
The stock market is a giant coordinated attempt to predict the economy, and it reached an all-time high on February 12, suggesting that analysts expected the economy to do great over the following few months. On February 20th it fell in a way that suggested a mild inconvenience to the economy, but it didn’t really start plummeting until mid-March – the same time the media finally got a clue. These aren’t empty suits on cable TV with no skin in the game. These are the best predictive institutions we have, and they got it wrong.
But… this isn't how it went down. As AllAmericanBreakfast and others pointed out in the comments, the market started reacting in the last week of February, with news headlines directly linking the decline to the ‘coronavirus’. By the time we get to mid-March, we’re not far off the bottom.
(You can confirm this for yourself in a few seconds by looking at a chart of the relevant time period.)
EDIT: Scott has explained his rationale here. Although I still think his version of events is incorrect as phrased, I want to make it clear I am not accusing him of deliberately massaging the data or any other such shenanigans, and the next paragraph about revisionist history etc was only meant to be a general observation about how people responded. My apologies to Scott for the unclear wording, as well any perceived slight against his very good reputation.
For whatever reason, COVID-19 seems to be a magnet for revisionist history and/or wishful thinking. In other comments under the same post, the notion that people from our ‘tribe’ did especially well also comes under serious question—in fact, it looks like many of the names mentioned seem to have jumped on the bandwagon after it was obvious, and certainly long after the market was moving.
The facts are that the market reacted faster than almost all of us. But not before a few prescient people placed their bets!
So now the question becomes: why didn’t the market react earlier than February 20, like those smart people did?
The null hypothesis is that the market reacted exactly appropriately on the basis of the information available. After all, there were other potential pandemics in the recent past that were successfully contained or eradicated.
On February 20, there were only 4 known cases in Italy. We were a long ways from the bloodbath that was coming. Maybe it was correct to move cautiously until further information came in?
EDIT: previously misattributed to AllAmericanBreakfast (who agrees with it).
[At that time] it may have been plausible for many people to think this would continue to play out like SARS – East Asia would solve their problem, everyone else would watch airport arrivals and quarantine them effectively, and within a few weeks everything would stabilize and gradually go away. By Feb. 27 it was clear that this wasn’t happening, since community spread was very clear from the public data in Italy and Iran, and probably also clear from genetic data in the United States and elsewhere.
So we reached a tipping point in those next few days, at which point, the market started responding more vigorously.
If the null hypothesis is true, then those early trades were not quite as prescient as they look. We might be making the mistake of ‘resulting’, and confusing the reality we ended up in with all the others which were possible at the time, in which those traders lost their shirts. It's really hard to have a useful object-level discussion about this, because these events are one-offs (this is the same argument we get into every year on Scott's predictions threads!) It's not like we can run the experiment again, and thank goodness for that.
Nevertheless, Wei Dai suggested that this was the final nail in the coffin of EMH—at least for him.
I want to pause here to give mad props to Wei Dai for being totally open about everything, and especially for doing the following:
- warning that options are dangerous, and you can easily lose the lot
- generic disclaimer about seeking financial advice
- updating the thread after he ended up losing 80% of his paper profits
- mentioning selection bias, and saying we'd be right to discount his evidence
Which only leaves the initial claim that "at least for me this puts a final nail in the coffin of EMH."
This is a polite way of hinting that you might be a brilliant investing wizard with the power to beat the market. Honestly, after making such a beautiful trade—and my gosh it really was beautiful—whom amongst us could resist that temptation? Certainly not me. And anyway, it might even be true!
In making sense of claims of this nature, the first thing we have to establish: what does it even mean to be able to beat the market?
Can Uncle George Beat the Market?
Uncle George really likes his new iPhone. Man, these things are nifty! The dancing poop emoji is hilarious. On the strength of this insight, George dials his broker and loads up on AAPL stock.the chosen one! the scourge of efficient markets! the stuff of eugene fama’s nightmares!
Over the next year, AAPL stock goes up 15 per cent, while the broader S&P 500 only goes up 10 per cent. George becomes insufferable at family dinners as he holds forth on his stock-picking powers. Guess the market isn’t so ‘efficient’ after all, huh. Suck it, Eugene Fama!
So: did Uncle George beat the market?
In the narrowest possible sense… yes.
In the sense in which we aim to string words together so that they mean things: no, of course not. By this definition, every single trade leads to one of the two parties ‘beating the market’. Millions of people beat the market while I wrote this sentence. I can flip a coin between Pepsi and Coke right now, and have a 50 per cent chance of becoming a market-beating genius.
The Uncle George example makes it glaringly obvious that a successful trade does not somehow ‘break’ efficient markets. And yet, this is the same naive criticism constantly leveled against the EMH: if the market moves in literally any direction, that must mean it was wrong before! My cousin who sold/bought before it went up/down beat the market!
Same goes for the Great Coronavirus Trade. The fact that some people got out of the market early is not even the tiniest bit surprising. Investors constantly think the market is going to crash, for any number of plausible reasons. This is the default state of affairs: we have successfully predicted 73 of the last five market crashes, etc.
These predictions are almost always wrong:
And almost all the people who make them would have been much better off taking the boring ‘buy and hold forever’ strategy.
But even a stopped clock is right twice a day. And of course, we’re much more likely to hear about the occasional brilliant successes than the near-constant dull failures.
So the most naive criticism of the EMH boils down to ‘it's possible to make a good trade’. This is just a property of trading. It tells us exactly nothing about the market’s efficiency, or lack thereof.
But some people really do beat the market—and not in the trivial sense. They're not merely stopped clocks, or highly visible 'survivors'. I’ll suggest a definition later on which strips out the effect of randomness.
Before we get there—doesn’t the concession that people can non-trivially beat the market already drive a stake through the EMH’s heart?
This is the second great misunderstanding: there is no conflict between the EMH and beating the market. That’s how the market gets efficient! You find an information asymmetry that isn’t priced in yet, and in exploiting it, you move the market a little further towards efficiency.
Let’s call this information asymmetry an ‘edge’.
If the EMH is true—or even just true-ish—that doesn’t mean the market can’t be beat. It means:
You shouldn’t expect to beat the market without a unique edge, except by chance
Now, this usually gets simplified down to ‘you can’t beat the market’. And most of the time, this simplification is good enough: you might get lucky and win in the Uncle George sense, but over an investing lifetime, you’ll almost certainly revert to the mean (which isn’t matching the market return—it’s underperforming it).
But if you can find some kind of edge, you really can win! So, what might a genuine edge look like?
The Uncle Georges of the world don’t have an edge. All of their thoughts have already been thunk by someone else (probably by millions of someone elses). Instead, their fortunes are entirely at the mercy of the myriad other forces that drive stock prices: consumer demand, workplace harassment scandals, money printers going brrr, the exact virulence of a novel coronavirus, the price of cheese in Spain last Friday afternoon, etc.
All of this stuff—billions of inputs processed by the greatest collective intelligence ever built—is a black box unto us mere mortals. It’s impossible to assign perfect causal explanations to stock prices, which means we can pick whichever story suits us best. As a market reporter, this was pretty much my whole job: calling brokers and economists to wrap a plausible narrative around totally inexplicable events, and generate sage nodding of heads.
All Uncle George can see is that he placed his bet, and AAPL went up. It was the poop emoji for sure!
And so, Uncle George spends his days dishing out hot stock tips on online forums, oblivious to the fact that his success was meaningless.[uncle georging intensifies]
What does a real edge look like?
A century ago, investors started noticing they could consistently pick up bargains by running very simple formulas over stock prices. The most famous is the ‘value investing’ approach developed by Ben Graham, and used by Warren Buffett and Charlie Munger. There was a genuine, big old inefficiency in the markets, and these guys had a great time exploiting it.
I think this might be the image most people have in their head when they think of ‘beating the market’—diligently studying The Intelligent Investor and learning about PE ratios or whatever.
But this is like trying to use a stone-age axe against a fighter jet. The Ben Graham information asymmetry has long since disappeared because…markets are efficient(ish)! Once the formula was widely known, it stopped working. Investors developed more sophisticated versions, more formulas, more pricing models. Once those got out, they stopped working too. Now there’s a great debate as to whether even the most complicated descendants of value might be totally dead. In which case, the anomaly has officially gone for good.
Either way, this is not how Buffett gets his edge, and it hasn’t been for decades. Here’s Munger:
The trouble with what I call the classic Ben Graham concept is that gradually the world wised up and those real obvious bargains disappeared. You could run your Geiger counter over the rubble and it wouldn’t click.
Buffett’s most brilliant achievement is weaving this folksy legend that he is a cute old grandpa who beats the market by backing the best companies. Let's take a look at how market-beating investors really make their money.
Modern Edges are Completely Ridiculousgreatest showman on earth
1. The Warren Buffett Halo Effect
In recent decades, Buffett has made a killing through juicy private deals which are completely out of reach of the average investor. Like, six billion dollar deals with three billion in preference rights and a guaranteed dividend. Like, lobbying the government to bail out the banks, then carving off a huge piece of the action. Like, being able to play around with Berkshire Hathaway’s $115 billion insurance float. Much of his fortune is built on taxpayer largesse.
Warren Buffett’s brand is so powerful that at this point, his success is a self-fulfilling prophecy: when Berkshire invests in a stock, everyone else piles in after him and drive the price up. Buffett even lends out his ‘halo’ to companies that need it—most famously during the GFC—so long as they give him a generous discount to the market price, of course. (Matt Levine has written some fascinating columns about this).
And yet, and yet… Berkshire Hathaway has underperformed for the last decade. Buffett would have been better off to take his own advice and put it all in index funds.
2. Hedge funds with armies of drones
There you are, sitting in your home office going through Walmart’s quarterly report and calculating PE ratios or whatever. Meanwhile, the professionals are using an army of drones to monitor the movement of shopping carts in Walmart parking lots in real time.
See also: sending foot soldiers out to every branch of a bakery chain at the close of business each day, because the numbered dockets start out at zero, and thus contain live sales data unavailable to the market.
And so, when renowned hedge fund manager Michael Steinhardt was asked the most important thing average investors could learn from him, here’s what he suggested:
“I’m their competition.”
And yet, and yet…almost all hedge funds underperform. Not all of them are trying to beat the market, but the tools at their disposal gives us a sense of the difficulty here.
3. High-frequency traders move mountains
If multiple people have access to the same information, the speed in which you can bring it to market also matters. So, we have high-frequency traders.
One firm spent $300 million laying a direct cable between Chicago to New Jersey. They cut straight through mountains and crossed rivers. The cable stretched 1331 kilometres. And they did this to shave four milliseconds off their transmission time.
And yet, and yet…microwaves came along and rendered the whole project obsolete. Trying to get an edge is expensive.
4. Being willing and able to commit felonies
Insider trading is a thing. See also: criminals who hack or otherwise steal sensitive private information.
And yet, and yet…even when criminals have advance access to earnings reports, they still don’t do all that well, which is evidence for the very strongest form of the EMH (the one that no-one, including me, believes can possibly be true).
So…what sort of edge do us lesser mortals have?
If we mumble something about having ‘good intuition’, or ‘subscribing to the Wall Street Journal' then we should consider the strong possibility that we are Uncle George.
If the answer involves ‘fundamental analysis’ or ‘Fibonacci retracements’, we're still in Uncle George territory. The only difference is that doing something complicated makes it easier to internally justify the belief that we know a secret no-one else does. But it is still (probably!) a mistaken belief.
The EMH Gets Stronger With Every Attack
So we know for sure that market-beating edges exist—I’ve even written them down for everyone to see!
I can only dream of possessing a halo effect so strong that everyone piles into a stock right after I announce I have graced it with my favour. I don’t have an army of drones at my command, or the ability to bore through mountains to shave milliseconds off my trading times, or a weekly round of golf with the CEO of a Fortune 500 company.
The market is never perfectly efficient. But relative to me, it might as well be.
Critics have pointed out plenty of cases in which the EMH doesn’t jive with reality—and they are absolutely right. So this is where it gets really weird.
The EMH is the only theory that grows stronger with every attack against it.
Every edge is constantly at risk of being gobbled up by an efficient-ish market. The ones I’ve mentioned can only be publicly knowledge because they're somewhat defensible: they’re based on personal relationships, capital investment, proprietary technology, etc. But they disappear too.
Most edges can’t even be spoken out loud without disappearing. If stocks systematically rise on the third Thursday of each month but only under a waxing moon, and then someone writes about it in public, you can kiss that anomaly goodbye. The EMH sucks it into its gigantic heaving maw, and it’s gone forever.
In other words: every time someone picks a hole in the theory and points out an inefficiency, they make the predictions generated by the EMH more robust! It’s like some freaky shoggoth thing that Just. Won’t. Die.you may not like it, but this is what peak efficiency looks like
Which gets us to the totally justified criticism of the theory: the only reason the EMH can pull this stunt is because it’s bullshit science.
It’s unfalsifiable! It responds to criticism by saying, ‘OK, good point, but now that I’ve factored that in, you should believe in my theory even more.’
And…we really should?
The only way I can think about the EMH without going insane is to remind myself that it generates a useful heuristic. It’s not a stable law, like we might find in hard sciences. It’s not perfectly accurate. At any given point in time, there are always competing models that do a better job of describing reality. But all those other models can stop working at any moment, with no warning! By the time you find out their predictive power is gone, it’s too late, and you probably lost a bunch of money! By contrast, the EMH is a reliable model—reliably vague and hand-wavy, yes, but also reliably useful.
We know there are inefficiencies in the market. In the fullness of time, they will be absorbed into the gelatinous alien-god’s hivemind. But before that happens, maybe we can make some money off of them.
So now we come to the final test. How do you tell if you’ve really found a market-beating edge—that is, a model of reality that has more predictive power than the EMH—or you’re fooling yourself like Uncle George?
If You’re so Smart, Why Aren’t You Rich?
Everyone knows a secret about themselves, or the people they know well, or can arbitrage some opportunity in a niche that few people are paying attention to. These illiquid private ‘markets’ are much more fertile hunting grounds for asymmetries, and something I encourage everyone to think about.
But the public security markets are a gigantic agglomeration of everyone’s predictions, which constantly hoovers up every new fragment of information, and recalibrates itself in real time. Your challenge is to try and predict why this giant meta-prediction is wrong, and in which direction.
If you think you can reliably beat, say, an index fund that passively tracks the S&P 500, this is a much stronger claim than it first appears.
For one thing, you’re claiming to be better than Warren Buffett, who has failed to pull this off in the last 10 years, despite his huge advantages, and has started saying the game is so hard that everyone should just buy index funds. But that’s nothing. You are also claiming that you have the power to beat the greatest collective intelligence humanity has ever created.
This is an extraordinary claim, and the thing about extraordinary claims is that they require extraordinary evidence.
Uncle George’s AAPL trade ain’t going to cut it. Here is the extraordinary evidence that I would personally want to see before agreeing that an investor can beat the market:
1. Big heaps of money
This is the one area of life where there really is no dodging that most venerable of sick burns: if you’re so smart, why aren’t you rich?
So the first piece of evidence I would accept is the fact that someone is very, very rich. Sitting atop a big old pile of cash. And they’d probably also open a hedge fund so they can take other people’s money and turn it into millions more.
2. Track record of outperformance
Maybe a genuine market-beater doesn’t have enough starting capital to make big piles of money on a relatively slim edge, and for some reason is unable to come up with any scheme to beg or borrow more? Or the anomaly is real, but disappears before they can get filthy rich?
In these scenarios, I would also accept a complete record of out-of-sample investment returns over time—no backtests! no selecting the best trades!—as compared against the appropriate risk-adjusted benchmark.
These evidential standards work both in the case that the EMH is ‘dead’, i.e. you can reliably beat the market using public domain info which ought to already be priced in, and in the case that it’s not, i.e. you really do need novel information to get an edge.
As for evidence that the EMH really is dead…hmm. It’s not a proper theory to begin with. But I guess it would be ‘dead’ when the predictions it generates stop being accurate or useful? Which would look something like: ‘finding out that plenty of people meet the standards above, despite having never been in possession of a scrap of information that wasn’t already available to the market’.
In doing so, we’d have to be very careful to make sure we aren’t just looking at the Uncle Georges who unwittingly drew the winning lottery ticket. After all, we should expect plenty of investors to beat the market for a long time—even for years on end—entirely by chance.
The Great Coin-Tossing Experiment
Say we held a national coin-flipping contest. After 15 rounds, one in every ~32,800 people would have managed to call every single toss correctly, perfectly predicting a sequence like this:
H T H H T T H H H H T H T T T
Pretty impressive, huh!
Well, only in a world where we don’t know about probability. In that world, we might mistake blind randomness for skill. The lucky few winners would be hailed as the heroes of their hometowns, do interviews with breathless breakfast TV hosts, and explain that it’s all about the precise flick of the wrist. Aspiring flippers would queue up to buy the inevitable best-selling book, Flip Me Off, and pay exorbitant sums for one-on-one coaching sessions with the master tossers.
Depressingly, this is exactly what happens in the world of investing. What does it mean to achieve the kind of success which only happens by chance with 0.0003 probability? In the United States alone, it means you end up with 10,000 lucky dopes who are indistinguishable from brilliant investors.
And fund managers don’t need to do anywhere near that well to attract a market-beating aura. They’re incentivised to swing for the fences, increasing the odds they beat the market in some highly visible fashion over some shorter period—say, a lucky season or two. They inevitably regress to the mean, sometimes crashing and burning in spectacular fashion, but it doesn’t matter so long as they manage to hose naive investors in the meantime.
We can never entirely rule out the effect of randomness—there will always be some tiny chance that Warren Buffett is really just the world’s greatest coin-flipper—but we have to draw the line somewhere, or the standard is impossibly high.
Once the odds of a fluke get pretty slim—someone is super duper rich, and they’ve made a ton of consistently good trades over time—I’d happily congratulate them on their market-beating prowess, give them all my money to invest, listen eagerly to their advice, etc.
Being good Bayesians, this is obviously a spectrum: if they are not at that point, but trending in the right direction, I would be less skeptical, etc. But the bar has to be pretty high, or there's no way to separate skill from randomness.
Scott and Eliezer have both alluded to their comments being informed by private information. Here's a reddit comment in which Scott responds to criticism of his 'EMH is the real victim' riff:
I think we're in an asymmetric position, in that I know these people pretty well, I know they've thought about efficient market before, and they're the sort of people I would expect to beat the market if anyone could. I agree that if you just hear some blogger say he saw some people beat the market once, that's not much evidence.
Eliezer definitely understands the EMH, because the descriptions of it in Inadequate Equilibria are among the most brilliant and insightful I've ever come across. And Scott is obviously super smart.
I would love to know what their evidential standards are, but I'm explicitly not calling them out, or any of the people mentioned earlier in the post. No-one is under the slightest obligation to share private evidence, and I would be thrilled if those folks were indeed the market-beating chosen ones.
But I am saying that people, in general, make these kind of claims all the time—in good faith and with no malicious intent—and in general, taking their advice is an extraordinarily bad idea.
A heuristic: if you (or someone you know) is confident they can beat the market, and yet you notice you are not sitting atop an enormous pile of wealth, it's at least worth considering the possibility that you might be fooling yourselves.
The Four Types of Investors
There are very obvious and well-known reasons why everyone loves to think they can beat the market: overconfidence, confirmation bias, ‘resulting’, selective memory, survivorship bias, etc.
These forces are so powerful that many people—myself included—blithely ignore the vast piles of evidence that suggest beating the market is incredibly difficult, and go ahead and try anyway. All of us think we are special, and (almost) all of us are wrong.
Even if we don't personally harbour this particular fantasy, there's also a natural tendency to want our tribe or our friends to be the brilliant visionaries who were ahead of the action, possess sweet market-beating skills, etc.
So we can roughly place investors into one of the following four quadrants:
('Losers' and 'winners' here is tongue-in-cheek, and not a value judgment: literally, losing/winning this game by either successfully beating the market, or failing to do so)
Deluded losers (‘Suckers’)
“Apple stock really is undervalued, but the market hasn’t recognised it yet. I just got unlucky—it was because of [elaborate rationalization]. Also, even if I got it wrong this time, I was really close. Next time!
“What’s that? Do I track my portfolio returns over time, and compare against the relevant risk-adjusted benchmark to see whether I’m actually outperforming? Well, there’s no need. I usually do pretty well for myself, and I’m expecting to improve—in fact, I just picked up this classic book called The Intelligent Investor…”
Deluded winners (‘Dumb Luck’)
“I knew Apple stock was undervalued! And I remember that other time I made a really good trade, too. Guess I’m pretty good at this game!
“…What’s that? I might have just got lucky? Hah, no. I even did the Fibonacci retracements and everything."
Realistic losers (‘Clear-Eyed Fools’)
“I keep a meticulous record of my portfolio returns, which forces me to acknowledge the fact that even though I occasionally do well, I am underperforming my benchmarks on a risk-adjusted basis. I am under no delusions about my prospects of finding an edge, and I know I really ought to take Warren Buffett’s advice and put all my money in index funds.
“But I enjoy playing the markets! It's like how a night in Vegas can have negative EV but still be positive utility, because of all the non-financial factors. So I’m gonna keep gambling with a small part of my portfolio, just for shits and giggles. In the event that I ‘win’, I will try really hard to resist the incredible internal pressure to start thinking of myself as a brilliant investing guru."
Realistic winners (‘Chosen One’)
“I keep a meticulous record of my portfolio returns, which have outperformed the appropriate risk-adjusted benchmarks to such a degree that I am confident I have found a genuine informational asymmetry. I will of course never tell anyone about it, or it will become useless.
“And I can never be entirely sure: it’s also possible that I just got lucky. But at the very least, I am sitting atop great piles of money, which is pretty nice.”
The vast majority of people who actively trade their account are ‘Suckers’. Some smaller number fall into the ‘Dumb Luck’ quadrant (Uncle George would stay there if he never places another trade, but he almost certainly won’t be able to help himself.)
The right-hand quadrants are much more sparsely populated. I guess there are a few ‘Clear-Eyed Losers’ floating around, and a tiny handful of ‘Chosen Ones’.
This rough distribution is probably not too controversial. The question is, which one are you?
(I made a poll on my website at this point in the post, just for a bit of fun: the results so far are brilliant)
Trying to Beat the Market is Like Crack for Smart People
There is a tendency for smart people to wander into areas they know very little about, and think they can do better than the actual experts who have years or decades of domain-specific knowledge, on the basis of being very smart, or having read some blog posts online about being more rational.
This would be OK if it was just a bit cringe. I love armchair pontificating as much as the next guy! The consequences are usually limited to mildly annoying the people who actually know what they’re talking about, and much eye-rolling when you triumphantly reinvent the wheel.
There is some upside too: reinventing the wheel is fun, because you get to, like, invent wheels. And very occasionally, it might even be true! No doubt smart outsiders are occasionally able to breeze into a new field and exploit some obvious inefficiencies.
But…oh boy. It’s really not true of this particular domain. And it’s not harmless either.
The central prediction generated by the EMH is that you should not expect to be able to beat the market (in the non-trivial sense) unless you have unique information or some similar edge.
This prediction is tested every day. We have great piles of evidence which suggest that it is correct: the vast majority of active investors do really badly.
Crucially, it’s not only regular schmucks who underperform. So do paid professionals, and active managers, and hedge funds, and all sorts of brilliant people who have made this their life’s work.
It's possible that I am not making many friends with this post. I certainly feel pretty nervous about publishing it. Everyone who thinks they can beat the market will have their hackles up! If it helps at all, I am not claiming the high ground. I have made every dumb investing mistake you could think of, and then a few more besides. I am painfully aware of how hard this is.
These days I would put myself in the ‘Clear-Eyed Fool’ quadrant, but only by a fingernail. It’s a constant battle even to stay there. I still do clever things that contradict my own boring advice, and annoyingly, am rewarded for my hubris just often enough to start entertaining the thought that I’m a brilliant investing genius after all. Then I force myself to calculate the IRR on my publicly-traded investments, and compare it against appropriate benchmarks, and manage to get a fingernail-hold back on boring old reality.
To the extent that I have succeeded as an investor, and I am doing quite nicely thank you, it has only come through forcing myself to acknowledge the main prediction that emerges from the very-much-alive-and-kicking EMH. The huge and underappreciated benefit of doing so is that I occasionally divert some of my attention elsewhere, to domains where I actually do have an edge—and then I win.
The EMH is a weird, counterintuitive, freaky-ass shoggoth of a thing and it still confuses the heck out of me, even after almost a decade of writing about finance. I almost certainly made some mistakes in the post—in the process of writing it, I noticed several ways in which my initial beliefs were subtly wrong, which has already been super useful for me, and helped me understand some of the (valid) objections people have raised.
So I would also like to use this post to open up the floor to any and all EMH discussion, and try to benefit from the smaller-but-still-powerful collective intelligence that is Less Wrong.
I'm going to add comments with some of my own questions and uncertainty. It would be nice to become less confused together, and try to get a better sense of where we should apply our efforts at the margin.
I think this post is missing the important part of actually doing this well / being a chosen one, from my perspective. That is, it seems to think of the EMH as something like an on/off switch, where either you think the market is better than you always and so you just blindly trust it, or you think you're better than the market always, and so you should be actively trading.
But my experience has been about every five years, an opportunity comes by and I think "oh, this is a major opportunity," and each time I've been right. [FWIW I didn't have this for COVID, because of a general plan to focus on work instead of the markets leading up to the pandemic, and then when I started thinking "oh this will actually be as bad as 1918" I was too busy trying to solve practical problems related to it that I didn't think seriously about the question of "should I be trading on this?"; I think the me that had thought about trading based off it would have mostly made the right calls.]
This is, of course, a small sample size, and I've made many active trades that weren't associated with that feeling, whose records have been much worse. Each of the 'edge' investments has also had another side to it, an... (read more)
I think I agree with this point and I'm glad you raised it—you might non-trivially beat the market every once in a while, but not all the time. To be clear, I don't think of the EMH as an on/off switch, and if the post gives that impression, that's my bad. In fact, the most successful investors I know of wait, wait, wait, and then very occasionally load up when they see a major asymmetry.
But I do still have a couple of reservations. The first is that this would still show up in the evidential standards I've suggested? If you match the market most of the time, but every five years you beat it, you're eventually gonna have a pretty amazing track record, especially as you compound early successes. Note this is more or less the same frequency offered by some anomalies—for example, the momentum strategy I experimented with only outperforms during downturns, but ends up with a super impressive CAGR.
The second question is, how confident are you ('you' in the general sense, but also 'you' Vaniver if you feel inclined to answer) that knowing the difference between the edge trades and the others actually helps? If it's a very st... (read more)
Historically the biggest short-term gains have been disproportionately amidst or immediately following bear markets, when volatility is highest.
Right, April's rally wasn't due to "actually, everything is great now", it was due to "whew, it looks like the most apocalyptic scenarios we were seeing in March aren't likely, and there's a limit to how bad it's going to get".
I agree with all of this. (And have had similar experiences.)
The opportunity feeling on the way in, and then something much closer to randomness on the way out in particular resonates, and I'm not sure I'd seen that actually written out as a general rule before.
EDIT: See also my comment here for some discussion of how I think this works. And see my reply to Eliezer here for some heuristics regarding when an opportunity might be real.
Thank you, I enjoyed this post.
One thing I would add is that the EMH also suggests one can make deviations that don't have very high EMH-predicted costs. Small investors do underperform indexes a lot by paying extra fees, churning with losses to spreads and capital gains taxes, spending time out of the market, and taking too much or too little over risk (and especially too much uncompensated risk from under diversification). But given the EMH they also can't actively pick equities with large expected underperformance. Otherwise, a hedge fund could make huge profits by just doing the opposite (they compete the mispricing down to a level where they earn normal profits). Reversed stupidity is not intelligence. [Edited paragraph to be clear that typical retail investors do severely underperform, just mainly for reasons other than uncanny ability to find overpriced securities and buy them).]
That consideration makes it more attractive, if one is uncertain about an edge, to consider investments that the EMH would predict should be have very modest underperformance, but some unusual information would suggest would outperform a lot. I was persuaded to deviate from indexing after s... (read more)
Seconding this. It turns out that investing under the current academic version of EMH (with time-varying risk premia and multifactor models) is a lot more complicated than putting one's money into an index fund. I'm still learning, but one thing even Carl didn't mention is that modern EMH is compatible with (even demands) certain forms of market timing, if your financial situation (mainly risk exposure through your non-investment income) differs from the average investor. This paper gives advice based on academic research but was apparently written in 1999 so may already be outdated.
I stumbled across a comment about efficient markets in an old Michael Vassar interview
Does anybody know what papers he's talking about? (I'm not sure if I transcribed the names properly.) They seem very relevant to this discussion.
He could be referring to:
De Long, J. B., Shleifer, A., Summers, L. H., & Waldmann, R. J. (1990). Noise trader risk in financial markets. Journal of political Economy, 98(4), 703-738. Retrieved from http://www.nccr-finrisk.uzh.ch/media/pdf/DeLongShleiferSummersWaldmann_JPE1990.pdf
From the abstract:
" The unpredictability of noise traders' beliefs creates a risk in the price of the asset that deters rational arbitrageurs from aggressively betting against them. As a result, prices can diverge significantly from fundamental values even in the absence of fundamental risk. Moreover, bearing a disproportionate amount of risk that they themselves create enables noise traders to earn a higher expected return than rational investors do."
(This paper has been quoted 6831 times according to Google Scholar).
I found out a couple of days ago that there's a debate within academia over EMH. Here's how John Cochrane described it in an interview:... (read more)
As you pointed out in the post, the version of the EMH you are describing, is unfalsifiable. To me it looks like an obvious misstep by our own rules, so why does it hold so much sway?
It's exactly the kind of thing we were warned about in the Mysterious Answers sequence: A Semantic Stopsign that stops you from asking the obvious next question. A Curiosity Stopper: because there are Smart People who know better, we are not socially obligated to even look.
I notice I am confused. If we're playing by the same epistemological rules, we should not have this kind of disagreement. At least one of us is missing something important. What is missing?
Its vagueness makes this EMH feel like a motte-and-bailey argument. Can you tell which part is the bailey and which is the motte? Think about this for at least ten seconds before reading on.
Here's a guess: "You can't hope to beat the market, so I don't have to try, and neither should you!" (bailey). But look at all these anomalies! "The market will price those in now that they are known. They'll evaporate. (Eventually.)" (motte).
But the vagueness means I can only guess. That's the nature of such arguments. Maybe the bailey is something else or ther... (read more)
Confusion: Is there some (incredibly weak) sense in which every person who makes a trade is bringing unique information to market?
Uncle George's opinion of the new iPhone is reflected in sales figures and aggregate reviews. But technically, he is in possession of a unique piece of private information: 'what does this one guy called Uncle George think about AAPL stock?'
This information is vanishingly close to being worthless, and a million miles way from a tradable edge, but it's not quite worthless—it still collectively helps to generate a signal. What does this mean, if anything?
Re: "revisionist history":
You criticize my description in "A Failure, But Not Of Prediction", which was:
As my post said, the market started declining a little in February. Using the S&P link you provide, on March 2/28, it reached 2954, just 12% lower than its all-time high, then quickly recovered to only 8% lower a few days later. For comparison, the market fell 7% in May 2019 because Donald Trump made a bad tweet, and then everyone laughed it off and forgot about it within a few weeks. I think my claim that "it fell in a way that suggested a mild inconvenience to the economy" is a fair description of this.
It had its next major fall on March 9, reaching a new low (34% off its all-time high) March 23. I think it is fair to say it started plumm... (read more)
I enjoyed both the perspective and the tone of the post. Up voted.
Two, probably not to deep thoughts.
First, years ago I had a discussion with some day traders and one was talking about the SPY EFT He mentioned that the fund owners actually did not invest in all 500 equities but only in the top performing ones (no idea what the criteria was on that). The argument was largely they could not make money running the fund if the invested in both the winning and losing stocks.
Second, and it relates to the first, one of the other things (different time) that was pointed out was "market" is a problematic terms. Each asset that is traded has a market and that is not the same as "the market". I think this tends to be something of a problem area for people when the issue of EMH is in question. Many (most) do okay in thinking about a market in APPL (supply and demand, buyers/sellers, bids and offers) we all get that. But "the market" is a much more complicated (even chaotic) animal. At a high level most also get general portfolio theory but probably could not put a customized plan in place for their personal portfolio. What I suspect happens here, with regard to EM... (read more)
For anyone else who wasn't aware:
Thanks, good post.
Three anecdotes about having an edge but only in illiquid markets where the derivable value is limited:
1. Back in 1991 a friend of mine had the following market-beating insight. He was watching Freddie Mercury on TV a few months before he died; at this point it was not publicly known he was ill with AIDS (nor even, bizarrely, that he was gay - I think all was only revealed just before he died).
My friend said "He's not looking well..." and then made a brilliant inference: "...so Bohemian Rhapsody will be the Christmas ... (read more)
I would discount it more if I hadn't if a random person had made those investments then Wei Dai. Wei Dai made previous very good investments in crypto and domain names with similar returns.
When thinking myself about investing in the middle of February I was thinking about advice on Wei Dai that's years old on LessWrong before actually talking with him about investing in this situation.
I think any of the high return investments of Wei Dai of crypto, domain names and... (read more)
I lost the thread when I got to the section "If You're so Smart, Why Aren't You Rich?". An assumption seems to become implicit there that if the EMH is false, then you can beat the market. But why is that so? I tend to think of the market as a random walk during day trading, largely driven by viral memetic fashion trends on the scale of weeks to months, and then often only "weighing" effectively on the scale of months to years.
Take the day trading example: the S&P had almost no cumulative returns during the day from 1993-... (read more)
Confusion: Can an anomaly be an open secret, and not defensible, and still somehow persist?
For example, I'm especially interested in the momentum anomaly. The momentum folks make a very compelling case that the behavioural econ reasons that brought it into existence are so dang powerful that it somehow persists, even though there are many papers/articles/books about it that anyone can read.
I personally found this just compelling enough to run a momentum experiment, based on a strategy that has done extremely well in reducing volatility and drawdowns i... (read more)
Could you expand on which momentum anomaly you tested? One type is cross-sectional momentum (buy the top 1/10th of stocks that went up and short the bottom 1/10th), which is subject to major periods of drastic underperformance. This is all well documented in the literature. The other type is using momentum on the market as a whole, perhaps switching between asset classes based on momentum. I would not think that you could assess a strategy based on 6 months of performance.
My view as an investor since the 1980s is that the EMH is true to a 0th approximation. However massive agency issues in the fund management industry leave room to outperform on a risk adjusted basis if your incentives are different from the average fund manager. Some anomalies are just not exploitable by fund managers/agents because they would lose all their funds under management after periods > 12 months of underperformance.
LW readers interested in the topic may like reading the Alphaarchitect blog. https://alphaarchitect.com/blog/
Here's a thought experiment: Suppose that a market is perfectly efficient, except that every 50 years or so there's a crash, which sufficiently smart people can predict a month in advance. Would you say that this market is efficient? Technically it isn't, because smart people have a systematic advantage over the market. But practically, no trader systematically beats the market, because no trader lives long enough!
I suppose you can create a long-living institution, a "black swan fund", that very rarely makes bets on predictable crashes, and over a few centuries can prove it has higher returns. But I guess not enough people care about returns over these timescales.
FWIW, I very much wanted to adjust my investments based on the covid news, but was stopped from doing so mainly via friction:
1. I made recent profits from my stocks, so selling them (even for a short window) would have had bad capital gains repercussions for me... in the US it's a PITA to deleverage stocks that are in the green.
2. I'm not enough of a trading wonk to feel comfortable buying puts, and the little in the past that I've played around with those tools seemed to indicate that it's hard for a non-professional to avoid excessive... (read more)
There is a simple explanation to this: It turns out that being a good rationalist with access to an option trading account is actually an edge.
I also realized that the market was wrong and told friends to pull out of equities in mid-February. They didn't listen.
Curated, this is a great and really grounding contribution to the ongoing conversation around this, it explains lots of key arguments really well. I hope to see more discussion about whether the EMH is dead and what can be learned from the covid situation, and I liked much of the comments section as well.
Something that occured to me while reading this post:
In a given situation, some people who are not at the bleeding edge of finance in general will be well placed to beat the market in a way we would tend to think of as due to skill rather than luck.
In the coinflip example, the people who correctly predict the sequence do so through luck. We call the sequence random because we are unable to model the process which generates it, and we call it luck when they are able to predict it because they don't know how to model it either. In the case of detecting... (read more)
I think one key question, when talking about the EMH, is what we mean for an information to be available.
It's plausible, that for the public it only became clear around 27. February that Covid19 would be huge.
But it seems some experts knew much earlier. Just a quick browse on epidemiological twitter, for example, and you can find quite some instances of people expecting this not to be contained in the beginning of February. There's also the case of a swiss epidemiologist who was one of the first to warn swiss national media about Covid19 and cla... (read more)
Confusion: Can multiple traders profit by bringing the exact same information to market?
I want to say yes? At least if they don't individually have enough heft to fully price it in? That would explain how multiple traders are able to exploit the same anomaly, whereas hedge funds have to cap their fund size because getting too big hurts them.
Thanks, I loved this post. Its obviously hard to distinguish luck from skill when it comes to investment returns, I don't think focusing on money and track record would be enough to convince me of skill. I think I would need to also rationally evaluate the investment strategy to determine if it can be reasonably expected to beat the market. Common strategies like "we do what everyone else does but better", or "we invest in value/growth" would probably not convince me (risk premiums, liquidity premiums, animal spirits probably affec... (read more)
Interesting post, would love to hear your opinion about my suggested alternative to EMH, I don't see anything in your post that contradicts it. But i'm having a very hard time accepting the idea that you can't beat the market using public data and infinite amount of computing power.
If the market regularly reacts to public statements of inefficiency ("stocks systematically rise on the third Thursday of each month but only under a waxing moon") by eliminating it, then this is *refutation* of the efficient market hypothesis.
Inefficiencies are priced in when they are: obvious, easy, and safe,. A non-obvious inefficiency is a true hidden edge. A non-easy inefficiency has a high barrier (capital, technical, physical, etc) to entry. A non-safe inefficiency has a high social barrier to entry.
Because names have power, I dub this the Passive Market Hypothesis.
In case you weren't aware, this point has been made here before, in a classic old post: Markets are Anti-Inductive.
I’ve never been a strict EMH believer. My definition is that you can’t find any free money anywhere. The market does obviously stupid things sometimes—see gamestop or amc recently. You can say what you want, but in my mind at least one of the meme stocks was for all intents and purposes objectively overvalued. But my definition is revealed by the state of the world currently—I am not rich because I saw something that was, in my mind, clearly wrong. See, even when GME was at it’s highs, the cost to borrow was over 100%, with the possibility of a squeeze goi... (read more)
I'm probably missing something - but when you say the vast majority of active investors do really badly, shouldn't that be impossible too? If markets are truly efficient, isn't it just as hard to underperform them as outperform?
You write quite entertainingly.
There's no room for luck in the "realistic" half of your diagram.
There were some sizable selloffs starting January 22nd.
Really excellent discussion of the EMH. Love your description of it as some kind of demon-god creature.
I'll jump in on a minor point.
This might instead be evidence that there is a lot more insider trading than we think -- making it so that those we know of were competing for gains against other insider traders.
Your second quote that you attributed to me was actually from keaswaran, who I agree with.
I think you're understating the amount of private information available to anyone with a reasonable level of intelligence. If you have a decent level of curiosity, chances are that you know some things that the rest of the world hasn't "caught onto" yet. For example, most fans of Tesla probably realized that EVs are going to kill ICEs and that Telsa is at least 4 years ahead of anyone else in terms of building EVs long before the sudden rise in Tesla stock in Jan 2020. Similarly, people who nerd out about epidemics predicted the scal... (read more)
Any antifragile systems is like this
EtA: Although I guess it's extremely (perfectly?) anti-fragile.
The efficient market says that the market price reflects "all available information". That's tautological, and it's hard to argue with tautologies. The problem is that trades are made by people or algorithms designed by people. Different people have different tolerances for risk, different ideas about market performance and different strategies for making money. What does it even mean for a price to reflect all available information? A lot of that information is embedded in people's minds and situations and in our institution struc... (read more)