Cross-posting from my personal blog, but written primarily for Less Wrong after recent discussion here.


There are whispers that the Efficient-Market Hypothesis is dead. Eliezer's faith has been shaken. Scott says EMH may have been the real victim of the coronavirus.

The EMH states that “asset prices reflect all available information”. The direct implication is that if you don’t have any non-available information, you shouldn’t expect to be able to beat the market, except by chance.

But some people were able to preempt the corona crash, without any special knowledge! Jacob mentioned selling some of his stocks before the market reacted. Wei Dai bought out-of-the-money 'put' options, and took a very handsome profit. Others shorted the market.

These people were reading the same news and reports as everyone else. They profited on the basis of public information that should have been priced in.

And so, the EMH is dead, or dying, or at the very least, has a very nasty-sounding cough.

I think that rumours of the death of efficient markets have been greatly exaggerated. It seems to me the EMH is very much alive-and-kicking, and the recent discussion often involves common misunderstandings that it might be helpful to iron out.

This necessarily involves pissing on people's parade, which is not much fun. So it's important to say upfront that although I don't know Wei Dai, he is no doubt a brilliant guy, that Jacob is my favourite blogger in the diaspora, that I would give my left testicle to have Scott's writing talent and ridiculous work ethic, that Eliezer is a legend whose work I have personally benefited from greatly, etc.

But in the spirit of the whole rationality thing, I want to gently challenge what looks more like a case of 'back-slaps for the boys' than a death knell for efficient markets.

First: how the heck did the market get the coronavirus so wrong?

The Great Coronavirus Trade

Lots of people initially underreacted to COVID-19. We are only human. But the stockmarket is not only human—it’s meant to be better than this.

Here’s Scott, in A Failure, But Not of Prediction:

The stock market is a giant coordinated attempt to predict the economy, and it reached an all-time high on February 12, suggesting that analysts expected the economy to do great over the following few months. On February 20th it fell in a way that suggested a mild inconvenience to the economy, but it didn’t really start plummeting until mid-March – the same time the media finally got a clue. These aren’t empty suits on cable TV with no skin in the game. These are the best predictive institutions we have, and they got it wrong.

But… this isn't how it went down. As AllAmericanBreakfast and others pointed out in the comments, the market started reacting in the last week of February, with news headlines directly linking the decline to the ‘coronavirus’. By the time we get to mid-March, we’re not far off the bottom.

(You can confirm this for yourself in a few seconds by looking at a chart of the relevant time period.)

EDIT: Scott has explained his rationale here. Although I still think his version of events is incorrect as phrased, I want to make it clear I am not accusing him of deliberately massaging the data or any other such shenanigans, and the next paragraph about revisionist history etc was only meant to be a general observation about how people responded. My apologies to Scott for the unclear wording, as well any perceived slight against his very good reputation.

For whatever reason, COVID-19 seems to be a magnet for revisionist history and/or wishful thinking. In other comments under the same post, the notion that people from our ‘tribe’ did especially well also comes under serious question—in fact, it looks like many of the names mentioned seem to have jumped on the bandwagon after it was obvious, and certainly long after the market was moving.

The facts are that the market reacted faster than almost all of us. But not before a few prescient people placed their bets!

So now the question becomes: why didn’t the market react earlier than February 20, like those smart people did?

The null hypothesis is that the market reacted exactly appropriately on the basis of the information available. After all, there were other potential pandemics in the recent past that were successfully contained or eradicated.

On February 20, there were only 4 known cases in Italy. We were a long ways from the bloodbath that was coming. Maybe it was correct to move cautiously until further information came in?

Here's keaswaran:

EDIT: previously misattributed to AllAmericanBreakfast (who agrees with it).

[At that time] it may have been plausible for many people to think this would continue to play out like SARS – East Asia would solve their problem, everyone else would watch airport arrivals and quarantine them effectively, and within a few weeks everything would stabilize and gradually go away. By Feb. 27 it was clear that this wasn’t happening, since community spread was very clear from the public data in Italy and Iran, and probably also clear from genetic data in the United States and elsewhere.

So we reached a tipping point in those next few days, at which point, the market started responding more vigorously.

If the null hypothesis is true, then those early trades were not quite as prescient as they look. We might be making the mistake of ‘resulting’, and confusing the reality we ended up in with all the others which were possible at the time, in which those traders lost their shirts. It's really hard to have a useful object-level discussion about this, because these events are one-offs (this is the same argument we get into every year on Scott's predictions threads!) It's not like we can run the experiment again, and thank goodness for that.

Nevertheless, Wei Dai suggested that this was the final nail in the coffin of EMH—at least for him.

I want to pause here to give mad props to Wei Dai for being totally open about everything, and especially for doing the following:

  • warning that options are dangerous, and you can easily lose the lot
  • generic disclaimer about seeking financial advice
  • updating the thread after he ended up losing 80% of his paper profits
  • mentioning selection bias, and saying we'd be right to discount his evidence

Which only leaves the initial claim that "at least for me this puts a final nail in the coffin of EMH."

This is a polite way of hinting that you might be a brilliant investing wizard with the power to beat the market. Honestly, after making such a beautiful trade—and my gosh it really was beautiful—whom amongst us could resist that temptation? Certainly not me. And anyway, it might even be true!

In making sense of claims of this nature, the first thing we have to establish: what does it even mean to be able to beat the market?

Can Uncle George Beat the Market?

Uncle George really likes his new iPhone. Man, these things are nifty! The dancing poop emoji is hilarious. On the strength of this insight, George dials his broker and loads up on AAPL stock.

the chosen one! the scourge of efficient markets! the stuff of eugene fama’s nightmares!

Over the next year, AAPL stock goes up 15 per cent, while the broader S&P 500 only goes up 10 per cent. George becomes insufferable at family dinners as he holds forth on his stock-picking powers. Guess the market isn’t so ‘efficient’ after all, huh. Suck it, Eugene Fama!

So: did Uncle George beat the market?

In the narrowest possible sense… yes.

In the sense in which we aim to string words together so that they mean things: no, of course not. By this definition, every single trade leads to one of the two parties ‘beating the market’. Millions of people beat the market while I wrote this sentence. I can flip a coin between Pepsi and Coke right now, and have a 50 per cent chance of becoming a market-beating genius.

The Uncle George example makes it glaringly obvious that a successful trade does not somehow ‘break’ efficient markets. And yet, this is the same naive criticism constantly leveled against the EMH: if the market moves in literally any direction, that must mean it was wrong before! My cousin who sold/bought before it went up/down beat the market!

Same goes for the Great Coronavirus Trade. The fact that some people got out of the market early is not even the tiniest bit surprising. Investors constantly think the market is going to crash, for any number of plausible reasons. This is the default state of affairs: we have successfully predicted 73 of the last five market crashes, etc.

These predictions are almost always wrong:

And almost all the people who make them would have been much better off taking the boring ‘buy and hold forever’ strategy.

But even a stopped clock is right twice a day. And of course, we’re much more likely to hear about the occasional brilliant successes than the near-constant dull failures.

So the most naive criticism of the EMH boils down to ‘it's possible to make a good trade’. This is just a property of trading. It tells us exactly nothing about the market’s efficiency, or lack thereof.

But some people really do beat the market—and not in the trivial sense. They're not merely stopped clocks, or highly visible 'survivors'. I’ll suggest a definition later on which strips out the effect of randomness.

Before we get there—doesn’t the concession that people can non-trivially beat the market already drive a stake through the EMH’s heart?

This is the second great misunderstanding: there is no conflict between the EMH and beating the market. That’s how the market gets efficient! You find an information asymmetry that isn’t priced in yet, and in exploiting it, you move the market a little further towards efficiency.

Let’s call this information asymmetry an ‘edge’.

If the EMH is true—or even just true-ish—that doesn’t mean the market can’t be beat. It means:

You shouldn’t expect to beat the market without a unique edge, except by chance

Now, this usually gets simplified down to ‘you can’t beat the market’. And most of the time, this simplification is good enough: you might get lucky and win in the Uncle George sense, but over an investing lifetime, you’ll almost certainly revert to the mean (which isn’t matching the market return—it’s underperforming it).

But if you can find some kind of edge, you really can win! So, what might a genuine edge look like?

Anomalies Exist!

The Uncle Georges of the world don’t have an edge. All of their thoughts have already been thunk by someone else (probably by millions of someone elses). Instead, their fortunes are entirely at the mercy of the myriad other forces that drive stock prices: consumer demand, workplace harassment scandals, money printers going brrr, the exact virulence of a novel coronavirus, the price of cheese in Spain last Friday afternoon, etc.

All of this stuff—billions of inputs processed by the greatest collective intelligence ever built—is a black box unto us mere mortals. It’s impossible to assign perfect causal explanations to stock prices, which means we can pick whichever story suits us best. As a market reporter, this was pretty much my whole job: calling brokers and economists to wrap a plausible narrative around totally inexplicable events, and generate sage nodding of heads.

All Uncle George can see is that he placed his bet, and AAPL went up. It was the poop emoji for sure!

And so, Uncle George spends his days dishing out hot stock tips on online forums, oblivious to the fact that his success was meaningless.

[uncle georging intensifies]

What does a real edge look like?

A century ago, investors started noticing they could consistently pick up bargains by running very simple formulas over stock prices. The most famous is the ‘value investing’ approach developed by Ben Graham, and used by Warren Buffett and Charlie Munger. There was a genuine, big old inefficiency in the markets, and these guys had a great time exploiting it.

I think this might be the image most people have in their head when they think of ‘beating the market’—diligently studying The Intelligent Investor and learning about PE ratios or whatever.

But this is like trying to use a stone-age axe against a fighter jet. The Ben Graham information asymmetry has long since disappeared because…markets are efficient(ish)! Once the formula was widely known, it stopped working. Investors developed more sophisticated versions, more formulas, more pricing models. Once those got out, they stopped working too. Now there’s a great debate as to whether even the most complicated descendants of value might be totally dead. In which case, the anomaly has officially gone for good.

Either way, this is not how Buffett gets his edge, and it hasn’t been for decades. Here’s Munger:

The trouble with what I call the classic Ben Graham concept is that gradually the world wised up and those real obvious bargains disappeared. You could run your Geiger counter over the rubble and it wouldn’t click.

Buffett’s most brilliant achievement is weaving this folksy legend that he is a cute old grandpa who beats the market by backing the best companies. Let's take a look at how market-beating investors really make their money.

Modern Edges are Completely Ridiculous

greatest showman on earth

1. The Warren Buffett Halo Effect

In recent decades, Buffett has made a killing through juicy private deals which are completely out of reach of the average investor. Like, six billion dollar deals with three billion in preference rights and a guaranteed dividend. Like, lobbying the government to bail out the banks, then carving off a huge piece of the action. Like, being able to play around with Berkshire Hathaway’s $115 billion insurance float. Much of his fortune is built on taxpayer largesse.

Warren Buffett’s brand is so powerful that at this point, his success is a self-fulfilling prophecy: when Berkshire invests in a stock, everyone else piles in after him and drive the price up. Buffett even lends out his ‘halo’ to companies that need it—most famously during the GFC—so long as they give him a generous discount to the market price, of course. (Matt Levine has written some fascinating columns about this).

And yet, and yet… Berkshire Hathaway has underperformed for the last decade. Buffett would have been better off to take his own advice and put it all in index funds.

2. Hedge funds with armies of drones

There you are, sitting in your home office going through Walmart’s quarterly report and calculating PE ratios or whatever. Meanwhile, the professionals are using an army of drones to monitor the movement of shopping carts in Walmart parking lots in real time.

See also: sending foot soldiers out to every branch of a bakery chain at the close of business each day, because the numbered dockets start out at zero, and thus contain live sales data unavailable to the market.

And so, when renowned hedge fund manager Michael Steinhardt was asked the most important thing average investors could learn from him, here’s what he suggested:

“I’m their competition.”

And yet, and yet…almost all hedge funds underperform. Not all of them are trying to beat the market, but the tools at their disposal gives us a sense of the difficulty here.

3. High-frequency traders move mountains

If multiple people have access to the same information, the speed in which you can bring it to market also matters. So, we have high-frequency traders.

One firm spent $300 million laying a direct cable between Chicago to New Jersey. They cut straight through mountains and crossed rivers. The cable stretched 1331 kilometres. And they did this to shave four milliseconds off their transmission time.

And yet, and yet…microwaves came along and rendered the whole project obsolete. Trying to get an edge is expensive.

4. Being willing and able to commit felonies

Insider trading is a thing. See also: criminals who hack or otherwise steal sensitive private information.

And yet, and yet…even when criminals have advance access to earnings reports, they still don’t do all that well, which is evidence for the very strongest form of the EMH (the one that no-one, including me, believes can possibly be true).

So…what sort of edge do us lesser mortals have?

If we mumble something about having ‘good intuition’, or ‘subscribing to the Wall Street Journal' then we should consider the strong possibility that we are Uncle George.

If the answer involves ‘fundamental analysis’ or ‘Fibonacci retracements’, we're still in Uncle George territory. The only difference is that doing something complicated makes it easier to internally justify the belief that we know a secret no-one else does. But it is still (probably!) a mistaken belief.

The EMH Gets Stronger With Every Attack

So we know for sure that market-beating edges exist—I’ve even written them down for everyone to see!

I can only dream of possessing a halo effect so strong that everyone piles into a stock right after I announce I have graced it with my favour. I don’t have an army of drones at my command, or the ability to bore through mountains to shave milliseconds off my trading times, or a weekly round of golf with the CEO of a Fortune 500 company.

The market is never perfectly efficient. But relative to me, it might as well be.

Critics have pointed out plenty of cases in which the EMH doesn’t jive with reality—and they are absolutely right. So this is where it gets really weird.

The EMH is the only theory that grows stronger with every attack against it.

Every edge is constantly at risk of being gobbled up by an efficient-ish market. The ones I’ve mentioned can only be publicly knowledge because they're somewhat defensible: they’re based on personal relationships, capital investment, proprietary technology, etc. But they disappear too.

Most edges can’t even be spoken out loud without disappearing. If stocks systematically rise on the third Thursday of each month but only under a waxing moon, and then someone writes about it in public, you can kiss that anomaly goodbye. The EMH sucks it into its gigantic heaving maw, and it’s gone forever.

In other words: every time someone picks a hole in the theory and points out an inefficiency, they make the predictions generated by the EMH more robust! It’s like some freaky shoggoth thing that Just. Won’t. Die.

you may not like it, but this is what peak efficiency looks like

Which gets us to the totally justified criticism of the theory: the only reason the EMH can pull this stunt is because it’s bullshit science.

It’s unfalsifiable! It responds to criticism by saying, ‘OK, good point, but now that I’ve factored that in, you should believe in my theory even more.’

And…we really should?

The only way I can think about the EMH without going insane is to remind myself that it generates a useful heuristic. It’s not a stable law, like we might find in hard sciences. It’s not perfectly accurate. At any given point in time, there are always competing models that do a better job of describing reality. But all those other models can stop working at any moment, with no warning! By the time you find out their predictive power is gone, it’s too late, and you probably lost a bunch of money! By contrast, the EMH is a reliable model—reliably vague and hand-wavy, yes, but also reliably useful.

We know there are inefficiencies in the market. In the fullness of time, they will be absorbed into the gelatinous alien-god’s hivemind. But before that happens, maybe we can make some money off of them.

So now we come to the final test. How do you tell if you’ve really found a market-beating edge—that is, a model of reality that has more predictive power than the EMH—or you’re fooling yourself like Uncle George?

If You’re so Smart, Why Aren’t You Rich?

Everyone knows a secret about themselves, or the people they know well, or can arbitrage some opportunity in a niche that few people are paying attention to. These illiquid private ‘markets’ are much more fertile hunting grounds for asymmetries, and something I encourage everyone to think about.

But the public security markets are a gigantic agglomeration of everyone’s predictions, which constantly hoovers up every new fragment of information, and recalibrates itself in real time. Your challenge is to try and predict why this giant meta-prediction is wrong, and in which direction.

If you think you can reliably beat, say, an index fund that passively tracks the S&P 500, this is a much stronger claim than it first appears.

For one thing, you’re claiming to be better than Warren Buffett, who has failed to pull this off in the last 10 years, despite his huge advantages, and has started saying the game is so hard that everyone should just buy index funds. But that’s nothing. You are also claiming that you have the power to beat the greatest collective intelligence humanity has ever created.

This is an extraordinary claim, and the thing about extraordinary claims is that they require extraordinary evidence.

Uncle George’s AAPL trade ain’t going to cut it. Here is the extraordinary evidence that I would personally want to see before agreeing that an investor can beat the market:

1. Big heaps of money

This is the one area of life where there really is no dodging that most venerable of sick burns: if you’re so smart, why aren’t you rich?

So the first piece of evidence I would accept is the fact that someone is very, very rich. Sitting atop a big old pile of cash. And they’d probably also open a hedge fund so they can take other people’s money and turn it into millions more.

2. Track record of outperformance

Maybe a genuine market-beater doesn’t have enough starting capital to make big piles of money on a relatively slim edge, and for some reason is unable to come up with any scheme to beg or borrow more? Or the anomaly is real, but disappears before they can get filthy rich?

In these scenarios, I would also accept a complete record of out-of-sample investment returns over time—no backtests! no selecting the best trades!—as compared against the appropriate risk-adjusted benchmark.

These evidential standards work both in the case that the EMH is ‘dead’, i.e. you can reliably beat the market using public domain info which ought to already be priced in, and in the case that it’s not, i.e. you really do need novel information to get an edge.

As for evidence that the EMH really is dead…hmm. It’s not a proper theory to begin with. But I guess it would be ‘dead’ when the predictions it generates stop being accurate or useful? Which would look something like: ‘finding out that plenty of people meet the standards above, despite having never been in possession of a scrap of information that wasn’t already available to the market’.

In doing so, we’d have to be very careful to make sure we aren’t just looking at the Uncle Georges who unwittingly drew the winning lottery ticket. After all, we should expect plenty of investors to beat the market for a long time—even for years on end—entirely by chance.

The Great Coin-Tossing Experiment

Say we held a national coin-flipping contest. After 15 rounds, one in every ~32,800 people would have managed to call every single toss correctly, perfectly predicting a sequence like this:

H T H H T T H H H H T H T T T

Pretty impressive, huh!

Well, only in a world where we don’t know about probability. In that world, we might mistake blind randomness for skill. The lucky few winners would be hailed as the heroes of their hometowns, do interviews with breathless breakfast TV hosts, and explain that it’s all about the precise flick of the wrist. Aspiring flippers would queue up to buy the inevitable best-selling book, Flip Me Off, and pay exorbitant sums for one-on-one coaching sessions with the master tossers.

Depressingly, this is exactly what happens in the world of investing. What does it mean to achieve the kind of success which only happens by chance with 0.0003 probability? In the United States alone, it means you end up with 10,000 lucky dopes who are indistinguishable from brilliant investors.

And fund managers don’t need to do anywhere near that well to attract a market-beating aura. They’re incentivised to swing for the fences, increasing the odds they beat the market in some highly visible fashion over some shorter period—say, a lucky season or two. They inevitably regress to the mean, sometimes crashing and burning in spectacular fashion, but it doesn’t matter so long as they manage to hose naive investors in the meantime.

We can never entirely rule out the effect of randomness—there will always be some tiny chance that Warren Buffett is really just the world’s greatest coin-flipper—but we have to draw the line somewhere, or the standard is impossibly high.

Once the odds of a fluke get pretty slim—someone is super duper rich, and they’ve made a ton of consistently good trades over time—I’d happily congratulate them on their market-beating prowess, give them all my money to invest, listen eagerly to their advice, etc.

Being good Bayesians, this is obviously a spectrum: if they are not at that point, but trending in the right direction, I would be less skeptical, etc. But the bar has to be pretty high, or there's no way to separate skill from randomness.

Scott and Eliezer have both alluded to their comments being informed by private information. Here's a reddit comment in which Scott responds to criticism of his 'EMH is the real victim' riff:

I think we're in an asymmetric position, in that I know these people pretty well, I know they've thought about efficient market before, and they're the sort of people I would expect to beat the market if anyone could. I agree that if you just hear some blogger say he saw some people beat the market once, that's not much evidence.

Eliezer definitely understands the EMH, because the descriptions of it in Inadequate Equilibria are among the most brilliant and insightful I've ever come across. And Scott is obviously super smart.

I would love to know what their evidential standards are, but I'm explicitly not calling them out, or any of the people mentioned earlier in the post. No-one is under the slightest obligation to share private evidence, and I would be thrilled if those folks were indeed the market-beating chosen ones.

But I am saying that people, in general, make these kind of claims all the time—in good faith and with no malicious intent—and in general, taking their advice is an extraordinarily bad idea.

A heuristic: if you (or someone you know) is confident they can beat the market, and yet you notice you are not sitting atop an enormous pile of wealth, it's at least worth considering the possibility that you might be fooling yourselves.

The Four Types of Investors

There are very obvious and well-known reasons why everyone loves to think they can beat the market: overconfidence, confirmation bias, ‘resulting’, selective memory, survivorship bias, etc.

These forces are so powerful that many people—myself included—blithely ignore the vast piles of evidence that suggest beating the market is incredibly difficult, and go ahead and try anyway. All of us think we are special, and (almost) all of us are wrong.

Even if we don't personally harbour this particular fantasy, there's also a natural tendency to want our tribe or our friends to be the brilliant visionaries who were ahead of the action, possess sweet market-beating skills, etc.

So we can roughly place investors into one of the following four quadrants:

('Losers' and 'winners' here is tongue-in-cheek, and not a value judgment: literally, losing/winning this game by either successfully beating the market, or failing to do so)

Deluded losers (‘Suckers’)

“Apple stock really is undervalued, but the market hasn’t recognised it yet. I just got unlucky—it was because of [elaborate rationalization]. Also, even if I got it wrong this time, I was really close. Next time!

“What’s that? Do I track my portfolio returns over time, and compare against the relevant risk-adjusted benchmark to see whether I’m actually outperforming? Well, there’s no need. I usually do pretty well for myself, and I’m expecting to improve—in fact, I just picked up this classic book called The Intelligent Investor…

Deluded winners (‘Dumb Luck’)

“I knew Apple stock was undervalued! And I remember that other time I made a really good trade, too. Guess I’m pretty good at this game!

“…What’s that? I might have just got lucky? Hah, no. I even did the Fibonacci retracements and everything."

Realistic losers (‘Clear-Eyed Fools’)

“I keep a meticulous record of my portfolio returns, which forces me to acknowledge the fact that even though I occasionally do well, I am underperforming my benchmarks on a risk-adjusted basis. I am under no delusions about my prospects of finding an edge, and I know I really ought to take Warren Buffett’s advice and put all my money in index funds.

“But I enjoy playing the markets! It's like how a night in Vegas can have negative EV but still be positive utility, because of all the non-financial factors. So I’m gonna keep gambling with a small part of my portfolio, just for shits and giggles. In the event that I ‘win’, I will try really hard to resist the incredible internal pressure to start thinking of myself as a brilliant investing guru."

Realistic winners (‘Chosen One’)

“I keep a meticulous record of my portfolio returns, which have outperformed the appropriate risk-adjusted benchmarks to such a degree that I am confident I have found a genuine informational asymmetry. I will of course never tell anyone about it, or it will become useless.

“And I can never be entirely sure: it’s also possible that I just got lucky. But at the very least, I am sitting atop great piles of money, which is pretty nice.”

***

The vast majority of people who actively trade their account are ‘Suckers’. Some smaller number fall into the ‘Dumb Luck’ quadrant (Uncle George would stay there if he never places another trade, but he almost certainly won’t be able to help himself.)

The right-hand quadrants are much more sparsely populated. I guess there are a few ‘Clear-Eyed Losers’ floating around, and a tiny handful of ‘Chosen Ones’.

This rough distribution is probably not too controversial. The question is, which one are you?

(I made a poll on my website at this point in the post, just for a bit of fun: the results so far are brilliant)

Trying to Beat the Market is Like Crack for Smart People

There is a tendency for smart people to wander into areas they know very little about, and think they can do better than the actual experts who have years or decades of domain-specific knowledge, on the basis of being very smart, or having read some blog posts online about being more rational.

This would be OK if it was just a bit cringe. I love armchair pontificating as much as the next guy! The consequences are usually limited to mildly annoying the people who actually know what they’re talking about, and much eye-rolling when you triumphantly reinvent the wheel.

There is some upside too: reinventing the wheel is fun, because you get to, like, invent wheels. And very occasionally, it might even be true! No doubt smart outsiders are occasionally able to breeze into a new field and exploit some obvious inefficiencies.

But…oh boy. It’s really not true of this particular domain. And it’s not harmless either.

The central prediction generated by the EMH is that you should not expect to be able to beat the market (in the non-trivial sense) unless you have unique information or some similar edge.

This prediction is tested every day. We have great piles of evidence which suggest that it is correct: the vast majority of active investors do really badly.

Crucially, it’s not only regular schmucks who underperform. So do paid professionals, and active managers, and hedge funds, and all sorts of brilliant people who have made this their life’s work.

It's possible that I am not making many friends with this post. I certainly feel pretty nervous about publishing it. Everyone who thinks they can beat the market will have their hackles up! If it helps at all, I am not claiming the high ground. I have made every dumb investing mistake you could think of, and then a few more besides. I am painfully aware of how hard this is.

These days I would put myself in the ‘Clear-Eyed Fool’ quadrant, but only by a fingernail. It’s a constant battle even to stay there. I still do clever things that contradict my own boring advice, and annoyingly, am rewarded for my hubris just often enough to start entertaining the thought that I’m a brilliant investing genius after all. Then I force myself to calculate the IRR on my publicly-traded investments, and compare it against appropriate benchmarks, and manage to get a fingernail-hold back on boring old reality.

To the extent that I have succeeded as an investor, and I am doing quite nicely thank you, it has only come through forcing myself to acknowledge the main prediction that emerges from the very-much-alive-and-kicking EMH. The huge and underappreciated benefit of doing so is that I occasionally divert some of my attention elsewhere, to domains where I actually do have an edge—and then I win.


Discussion

The EMH is a weird, counterintuitive, freaky-ass shoggoth of a thing and it still confuses the heck out of me, even after almost a decade of writing about finance. I almost certainly made some mistakes in the post—in the process of writing it, I noticed several ways in which my initial beliefs were subtly wrong, which has already been super useful for me, and helped me understand some of the (valid) objections people have raised.

So I would also like to use this post to open up the floor to any and all EMH discussion, and try to benefit from the smaller-but-still-powerful collective intelligence that is Less Wrong.

I'm going to add comments with some of my own questions and uncertainty. It would be nice to become less confused together, and try to get a better sense of where we should apply our efforts at the margin.

150

85 comments, sorted by Highlighting new comments since Today at 5:56 AM
New Comment
Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings

I think this post is missing the important part of actually doing this well / being a chosen one, from my perspective. That is, it seems to think of the EMH as something like an on/off switch, where either you think the market is better than you always and so you just blindly trust it, or you think you're better than the market always, and so you should be actively trading.

But my experience has been about every five years, an opportunity comes by and I think "oh, this is a major opportunity," and each time I've been right. [FWIW I didn't have this for COVID, because of a general plan to focus on work instead of the markets leading up to the pandemic, and then when I started thinking "oh this will actually be as bad as 1918" I was too busy trying to solve practical problems related to it that I didn't think seriously about the question of "should I be trading on this?"; I think the me that had thought about trading based off it would have mostly made the right calls.]

This is, of course, a small sample size, and I've made many active trades that weren't associated with that feeling, whose records have been much worse. Each of the 'edge' investments has also had another side to it, an

... (read more)

I think I agree with this point and I'm glad you raised it—you might non-trivially beat the market every once in a while, but not all the time. To be clear, I don't think of the EMH as an on/off switch, and if the post gives that impression, that's my bad. In fact, the most successful investors I know of wait, wait, wait, and then very occasionally load up when they see a major asymmetry.

But I do still have a couple of reservations. The first is that this would still show up in the evidential standards I've suggested? If you match the market most of the time, but every five years you beat it, you're eventually gonna have a pretty amazing track record, especially as you compound early successes. Note this is more or less the same frequency offered by some anomalies—for example, the momentum strategy I experimented with only outperforms during downturns, but ends up with a super impressive CAGR.

The second question is, how confident are you ('you' in the general sense, but also 'you' Vaniver if you feel inclined to answer) that knowing the difference between the edge trades and the others actually helps? If it's a very st... (read more)

April was the stock market's best month in 30 years, which is not really what you expect during a global pandemic.

Historically the biggest short-term gains have been disproportionately amidst or immediately following bear markets, when volatility is highest.

9orthonormal6moRight, April's rally wasn't due to "actually, everything is great now", it was due to "whew, it looks like the most apocalyptic scenarios we were seeing in March aren't likely, and there's a limit to how bad it's going to get".

I agree with all of this. (And have had similar experiences.)

The opportunity feeling on the way in, and then something much closer to randomness on the way out in particular resonates, and I'm not sure I'd seen that actually written out as a general rule before.

EDIT: See also my comment here for some discussion of how I think this works. And see my reply to Eliezer here for some heuristics regarding when an opportunity might be real.

Thank you, I enjoyed this post.

One thing I would add is that the EMH also suggests one can make deviations that don't have very high EMH-predicted costs. Small investors do underperform indexes a lot by paying extra fees, churning with losses to spreads and capital gains taxes, spending time out of the market, and taking too much or too little over risk (and especially too much uncompensated risk from under diversification). But given the EMH they also can't actively pick equities with large expected underperformance. Otherwise, a hedge fund could make huge profits by just doing the opposite (they compete the mispricing down to a level where they earn normal profits). Reversed stupidity is not intelligence. [Edited paragraph to be clear that typical retail investors do severely underperform, just mainly for reasons other than uncanny ability to find overpriced securities and buy them).]

That consideration makes it more attractive, if one is uncertain about an edge, to consider investments that the EMH would predict should be have very modest underperformance, but some unusual information would suggest would outperform a lot. I was persuaded to deviate from indexing after s... (read more)

I’d also flag that going all-in on EMH and modern financial theory still leads to fairly unusual investing behavior for a retail investor, moreso than I had thought before delving into it.

Seconding this. It turns out that investing under the current academic version of EMH (with time-varying risk premia and multifactor models) is a lot more complicated than putting one's money into an index fund. I'm still learning, but one thing even Carl didn't mention is that modern EMH is compatible with (even demands) certain forms of market timing, if your financial situation (mainly risk exposure through your non-investment income) differs from the average investor. This paper gives advice based on academic research but was apparently written in 1999 so may already be outdated.

5ChristianKl7moIt doesn't suggest that. Factually, we know that a majority of investors underperform indexes. When there's an event that will cause retail investors to predictively make bad investments some hedge fund will do high frequency trades as soon the event becomes known to be able to trade the opposite site of the trade. All events that cause more retail investors to buy a stock then it cause retail investors to sell the stock needs some hedge fund or bank to take the opposite side of the trade and likely that hedge fund or bank is in the trade because it has models that suggest it's a good trade for them. A hedge fund that provides liquity to trades is going to make as much money under EMH as it cost to do the market making when it competes with other hedge funds. It worth noting that a targeted date index fund does make predictable trades where someone needs to do the market making and will likely make a small profit for doing the market making.
5CarlShulman7moAbsolutely, I mean that when you break out the causes of the underperformance, you can see how much is from spending time out of the market, from paying high fees, from excessive trading to pay spreads and capital gains taxes repeatedly, from retail investors not starting with all their future earnings invested (e.g. often a huge factor in the Dalbar studies commonly cited to sell high fee mutual funds to retail investors), and how much from unwittingly identifying overpriced securities and buying them. And the last chunk is small relative to the rest. I agree, active investors correcting retail investors can earn normal profits on the EMH, and certainly market makers get spreads. But competition is strong, and spreads have been shrinking, so that's much less damaging than identifying seriously overpriced stocks and buying them.
3ChristianKl7moActive investors need to spend money to hire analysts, build computer models and high-frequency trading computers. Let's say it costs $10 dollar/per trade to do the analysis to be able to do a trade with a retail investor that nets the hedge fund $10.10. Even when there's no strong competition with other hedge funds over that $0.10 of profit, the retail investor is still screwed by a significant $10.10.
3Thomas Kwa7moDo we know that this isn't currently happening, i.e. that observing what retail investors buy and betting against them isn't a major profit stream for hedge funds?
3CarlShulman7moSure, it's part of how they earn money, but competition between them limits what's left, since they're bidding against each other to take the other side from the retail investor, who buys from or sells to the hedge fund offering the best deal at the time (made somewhat worse by deadweight losses from investing in speed).

I found out a couple of days ago that there's a debate within academia over EMH. Here's how John Cochrane described it in an interview:

When house prices are high relative to rents, when stock prices are high relative to earnings—that seems to signal a period of low returns. When prices are high relative to earnings, it’s not going to be a great time to invest over the next seven to ten years. That’s a fact. It took us ten years to figure it out, but that’s what (Robert) Shiller’s volatility stuff was about; it is what Gene (Fama)’s regressions in the nineteen-eighties were about. That was a stunning new fact. Before, we would have guessed that prices high relative to earnings means we are going to see great growth in earnings. It turned out to be the opposite. We all agree on the fact. If prices are high relative to earnings that means this is going to be a bad ten years for stocks. It doesn’t reliably predict a crash, just a period of low returns, which sometimes includes a crash, but sometimes not.

Ok, this is the one and only fact in this debate. So what do we say about that? Well, one side says that people were irrationally optimistic. The other side says, wait a minute, the t

... (read more)

I stumbled across a comment about efficient markets in an old Michael Vassar interview

I think that when when I look at economics (another great example) you have a series of papers by Larry Summers and Brad DeLong in the early 90s that as far as I can tell drive a stake through the heart of the idea of efficient markets. They show that if you make the extremely minimal assumption of people not being perfectly capable of assessing how much risk is involved in an investment, there should be a systematic tendency for markets to become less efficient with time.

And as far as I can tell this behavior--this paper despite being done by pretty much the top people in economics--just got ignored and had no impact on the--or this series of papers--had no impact on the progression of the field. It was logically ironclad. That's the sort of thing I basically expect from most sciences in the modern world--almost everything but applied physics--and it's (you know) this is a particularly clear case though because you have essentially the strongest possible argument done by the most prestigious possible people with just no recollection of it ever having happened in the profession.

Does anybody know what papers he's talking about? (I'm not sure if I transcribed the names properly.) They seem very relevant to this discussion.

He could be referring to:

De Long, J. B., Shleifer, A., Summers, L. H., & Waldmann, R. J. (1990). Noise trader risk in financial markets. Journal of political Economy, 98(4), 703-738. Retrieved from http://www.nccr-finrisk.uzh.ch/media/pdf/DeLongShleiferSummersWaldmann_JPE1990.pdf

From the abstract:

" The unpredictability of noise traders' beliefs creates a risk in the price of the asset that deters rational arbitrageurs from aggressively betting against them. As a result, prices can diverge significantly from fundamental values even in the absence of fundamental risk. Moreover, bearing a disproportionate amount of risk that they themselves create enables noise traders to earn a higher expected return than rational investors do."

(This paper has been quoted 6831 times according to Google Scholar).

8MTGandP6moA different paper but in the same vein: Markets are efficient if and only if P= NP [https://arxiv.org/pdf/1002.2284.pdf]
7gilch6moI upvoted for relevance (although it's not quite what I asked for). Interesting read. The efficient-market hypothesis is coherent, falsifiable and wrong [https://www.greaterwrong.com/posts/u6JzcFtPGiznFgDxP/excluding-the-supernatural] , even in its weakest form [https://en.wikipedia.org/wiki/Efficient-market_hypothesis#Weak,_Semi-Strong,_and_Strong-Form_Tests] : The paper describes a way to introduce an inefficiency, that were it arbitraged optimally, would solve an (NP-complete) satisfiability problem. Even if P = NP (which seems unlikely), it remains unproven, so nobody yet knows how to solve NP-complete problems in PTIME, which would be required for markets to be truly efficient. The eldritch abomination may grow stronger with every attack, but it is decaying even faster than we can fight it. The number of possibilities to search for anomalies grows faster than our ability to compute them when you account for the possibility of anomalies in the pricing of sets of securities, like for pairs trading [https://en.wikipedia.org/wiki/Pairs_trade]. There might be something like 100,000 publicly-traded stocks in the world, but that would mean (1000002) combinations or nearly five billion pairs. And that's not even considering larger subsets, like triples, which would be on the order of hundreds of trillions. So the EMH can't be literally true. but can it be approximately true? It must be to some degree for the question to be this contentious. The real question I'm interested in, is "Are the markets exploitable?" Is there such a thing as alpha [https://en.wikipedia.org/wiki/Alpha_(finance)], or its it just luck? If finding exploits is as expensive as exploiting them is profitable, then the EMH might as well be true for that purpose. But if there are plenty of anomalies to go around, then I could have a realistic chance of finding and profiting from one nobody has noticed before.
4stoat6moI think The Economic Consequences of Noise Traders [https://www.nber.org/papers/w2395] is one of those papers.

As you pointed out in the post, the version of the EMH you are describing, is unfalsifiable. To me it looks like an obvious misstep by our own rules, so why does it hold so much sway?

It's exactly the kind of thing we were warned about in the Mysterious Answers sequence: A Semantic Stopsign that stops you from asking the obvious next question. A Curiosity Stopper: because there are Smart People who know better, we are not socially obligated to even look.

I notice I am confused. If we're playing by the same epistemological rules, we should not have this kind of disagreement. At least one of us is missing something important. What is missing?

Its vagueness makes this EMH feel like a motte-and-bailey argument. Can you tell which part is the bailey and which is the motte? Think about this for at least ten seconds before reading on.

Here's a guess: "You can't hope to beat the market, so I don't have to try, and neither should you!" (bailey). But look at all these anomalies! "The market will price those in now that they are known. They'll evaporate. (Eventually.)" (motte).

But the vagueness means I can only guess. That's the nature of such arguments. Maybe the bailey is something else or ther

... (read more)

Re: "revisionist history":

You criticize my description in "A Failure, But Not Of Prediction", which was:

The stock market is a giant coordinated attempt to predict the economy, and it reached an all-time high on February 12, suggesting that analysts expected the economy to do great over the following few months. On February 20th it fell in a way that suggested a mild inconvenience to the economy, but it didn’t really start plummeting until mid-March – the same time the media finally got a clue.

As my post said, the market started declining a little in February. Using the S&P link you provide, on March 2/28, it reached 2954, just 12% lower than its all-time high, then quickly recovered to only 8% lower a few days later. For comparison, the market fell 7% in May 2019 because Donald Trump made a bad tweet, and then everyone laughed it off and forgot about it within a few weeks. I think my claim that "it fell in a way that suggested a mild inconvenience to the economy" is a fair description of this.

It had its next major fall on March 9, reaching a new low (34% off its all-time high) March 23. I think it is fair to say it started plumm... (read more)

5Richard Meadows7moHi Scott, If we take the 6th of March—the last trading day before the March 9 fall—then the market is down 12.2%, which is already in 'correction' territory, and an extremely rapid descent by historical standards. If we take the 16th of March—the closest trading day we have to 'mid-March'—the market is already down 29.5% per cent, which is not too far off the bottom, and well and truly into 'bear' territory. I thought the talk about EMH being dead was weird at the time, and left a comment saying as much. I also wrote a post on March 24, which later turned out to be the bottom, saying that timing the market was a really bad idea, and the buy-and-hold forever strategy was about the best anyone could hope for. I am as surprised about the speed of the rebound as anyone! I possess no predictive powers, but I have consistently been defending boring orthodoxy. I admire and respect you very much as a person and a thinker—seriously man, you have no idea—so I feel extremely bad if I have made you feel bad. I didn't mean to accuse you specifically of being a revisionist historian—it was more of a general vibe that seemed to be happening a lot—and although I think the passage as stated is misleading, I don't think it's deliberately so, and I've edited the post so it comes off as less accusatory.

Confusion: Is there some (incredibly weak) sense in which every person who makes a trade is bringing unique information to market?

Uncle George's opinion of the new iPhone is reflected in sales figures and aggregate reviews. But technically, he is in possession of a unique piece of private information: 'what does this one guy called Uncle George think about AAPL stock?'

This information is vanishingly close to being worthless, and a million miles way from a tradable edge, but it's not quite worthless—it still collectively helps to generate a signal. What does this mean, if anything?

3ChristianKl7moThe question is whether or not Uncle George is worse or better informed then the professional investors. In situation like this I think: In early February I could have reasonably said: I think that should be your mental model when doing most trades that aren't index based.
3ESRogs7moSeems like the answer to this question is straightforwardly yes. People's desires to hold various assets will be super correlated with each other. But there's non-zero information in each person's preferences.
6jmh7moI'm pretty sure there is an entire literature (but cannot know think of the term) regarding the problem if everyone were to be index investors. Basically, if no one is investing in the underlying assets, in the extreme, we don't get pricing for those assets and ultimately cannot even price the indexes well. In short, the market implodes on itself without all those individual market interactions.
4Richard Meadows7moYeah, active investors are providing a valuable service to everyone else, both by exploiting genuine asymmetries, and by collectively generating a signal in the Uncle George sense. People sometimes worry about the passive revolution for this reason, but the vast majority of funds under management are still active, and human nature being what it is, there will presumably always be plenty of people willing to have a crack.
5ESRogs7moIs this true? A quick search suggests not: https://www.morningstar.com/insights/2019/06/12/asset-parity [https://www.morningstar.com/insights/2019/06/12/asset-parity]
6Richard Meadows7moHuh, I got that from a recent Bloomberg article [https://www.bloomberg.com/opinion/articles/2019-10-01/passive-investing-vs-active-investing-which-is-bigger] which says 15:1...not sure who's right or why the numbers are so different.
1fuego6moI think the reason not to worry too much about the passive revolution is that as long as there are money-making opportunities from active trading -- people will be incentivized to do it. The end state "everyone is passive -- there is no price signal" is not one we can reach from where we are, and it is not a stable equilibrium. If we ever did cross over into "too few hedge funds", there would such a strong incentive for more, that I don't think it would last long. But perhaps I misunderstand this critique.
2ChristianKl7moWe do get a price for those assets but that price is inefficient. That means that there's an opportunity for market participants to make money of the inefficiency.

I enjoyed both the perspective and the tone of the post. Up voted.

Two, probably not to deep thoughts.

First, years ago I had a discussion with some day traders and one was talking about the SPY EFT He mentioned that the fund owners actually did not invest in all 500 equities but only in the top performing ones (no idea what the criteria was on that). The argument was largely they could not make money running the fund if the invested in both the winning and losing stocks.

Second, and it relates to the first, one of the other things (different time) that was pointed out was "market" is a problematic terms. Each asset that is traded has a market and that is not the same as "the market". I think this tends to be something of a problem area for people when the issue of EMH is in question. Many (most) do okay in thinking about a market in APPL (supply and demand, buyers/sellers, bids and offers) we all get that. But "the market" is a much more complicated (even chaotic) animal. At a high level most also get general portfolio theory but probably could not put a customized plan in place for their personal portfolio. What I suspect happens here, with regard to EM... (read more)

4Richard Meadows7moYes! This is another really great point. I think Noah Smith described it by saying something like, 'there is no EMH—there are an infinite number of EMHs all happening at the same time'. Which is another reason the theory is vague and unfalsifiable. I find it helpful to think about each market in the narrowest possible terms. For example, the market for AAPL stock is likely to be less efficient than the S&P 500 market as a whole, although presumably not by much. The market for a thinly-traded stock languishing on the secondary board of the Tanzanian stock exchange is likely to be much less efficient than that. The market for a private company with no disclosure obligations is much less efficient again. By the time you get down to 'the market for the time and labour of this one guy called Richard', there are truckloads of inefficiencies and EMH doesn't really apply.

Thanks, good post.

Three anecdotes about having an edge but only in illiquid markets where the derivable value is limited:

1. Back in 1991 a friend of mine had the following market-beating insight. He was watching Freddie Mercury on TV a few months before he died; at this point it was not publicly known he was ill with AIDS (nor even, bizarrely, that he was gay - I think all was only revealed just before he died).

My friend said "He's not looking well..." and then made a brilliant inference: "...so Bohemian Rhapsody will be the Christmas ... (read more)

3orthonormal6moRight, the EMH doesn't fully apply when sharks can't swoop in with bets large enough to overwhelm the confederacy of Georges. The odds bookies offer are a hybrid between a market and a democracy.
1Richard Meadows7moThanks for sharing your experience—the third section in particular is really interesting. Relative to the central discussion: how much time and effort did you have to put in to find those edges, and do you think it was worth it? Would you encourage others to try and do something similar? Or is it more like a hobby?
3bfinn7moIt's probably taken about a year's full-time work (since 2006). Though I could have spent much less time by using a simpler albeit less effective version of what I do (as indeed I did initially) - just that I like improving it and investigating new things that might work. It is kind of a hobby, but involving a sufficiently serious amount of time and money and risk to be rather more than that. I think it probably has been worth it, as I reckon the value of my edge is rather bigger than my time cost in salary equivalent, though it is hard to be sure because of the large variability in returns (not least because of two big crashes since I started). But actually I wouldn't encourage others to try. There can't be many opportunities to beat the market, not many people can risk big enough bets to justify the time taken finding them, and there's a lot to be said for adopting a simple mechanical system that takes little time; maybe just buy & hold (though maybe a bit fancier than that). Not sure it's worth trying to pick stocks (I almost never do that) - just buy an index; though there is some fun to be had doing so, like picking horses, and kidding yourself you have special insight!
Aten't

For anyone else who wasn't aware:

The way that Granny Weatherwax, in Discworld, spells ain't on her sign to inform people that she 'aten't dead'.

https://www.urbandictionary.com/define.php?term=aten%27t

mentioning selection bias, and saying we'd be right to discount his evidence

I would discount it more if I hadn't if a random person had made those investments then Wei Dai. Wei Dai made previous very good investments in crypto and domain names with similar returns.

When thinking myself about investing in the middle of February I was thinking about advice on Wei Dai that's years old on LessWrong before actually talking with him about investing in this situation.

I think any of the high return investments of Wei Dai of crypto, domain names and... (read more)

4Wei_Dai7moSee also the bottom of this comment [https://ea.greaterwrong.com/posts/g4oGNGwAoDwyMAJSB/how-much-leverage-should-altruists-use/comment/RBXqgYshRhCJsCvWG] for a more complete record of my significant (non-EMH) investments.
4Richard Meadows7moCrypto and domain names might be in a different class, for sure—I don't know how those markets work, although I'd be interested to hear more. But options contracts that involve e.g. the fortunes of the S&P 500 are not. (I want to encourage rationalists to look for future opportunities to trade on, too. I just think those opportunities are vanishingly unlikely to come from highly liquid securities markets, and a lot of the recent talk is going to lead to misplaced effort.) Yeah, I'm still not exactly sure about whether anomalies can persist in public (see one of my confusions below). But if the trade is based on a one-off event, like this one, it's obviously fine to tell people about it. If it's a formula for an ongoing edge that persists in [stock-picking/domain name-picking/crypto] then not so much.
2ChristianKl7moI think that when it comes to events that have such high societal impact as COVID-19 and that don't really fall in anyone's expertise that happen in the future it's likely that rationalists can lucratively trade on them and pre-run the market. These kinds of event might be once per decade, but when they come along it's worth trading on them. As far as domain name picking goes, Wei Dai brought WeiDai.com at a time where US domains did cost money but there Chinese domain names weren't worth much and sold it years later. The edge isn't in trading either of that in the current market. In hindsight mispricing of Chinese domain and Crypto exist for years while being public. I think there was maybe 1-2 weeks of mispriced pandemic information in the case of COVID-19 when I take my personal information at the time into account. In neither of those cases it would have changed much to be public about one's investment about how long the anomalies persisted.

I lost the thread when I got to the section "If You're so Smart, Why Aren't You Rich?". An assumption seems to become implicit there that if the EMH is false, then you can beat the market. But why is that so? I tend to think of the market as a random walk during day trading, largely driven by viral memetic fashion trends on the scale of weeks to months, and then often only "weighing" effectively on the scale of months to years.

Take the day trading example: the S&P had almost no cumulative returns during the day from 1993-... (read more)

2ESRogs4moIs this meant to be in contrast to what a believer in the EMH would think? It sounds pretty similar to me. Is your point just that the information that the market is pricing in on short timescales more about the demand to hold various assets than new fundamental information about a company? I suppose if you think of the EMH as saying that the market only moves on fundamentals then that would be a contrast. But I guess I tend to think of the EMH as saying that all the fundamentals are priced in. Not that nothing else is priced in.

Confusion: Can an anomaly be an open secret, and not defensible, and still somehow persist?

For example, I'm especially interested in the momentum anomaly. The momentum folks make a very compelling case that the behavioural econ reasons that brought it into existence are so dang powerful that it somehow persists, even though there are many papers/articles/books about it that anyone can read.

I personally found this just compelling enough to run a momentum experiment, based on a strategy that has done extremely well in reducing volatility and drawdowns i... (read more)

Could you expand on which momentum anomaly you tested? One type is cross-sectional momentum (buy the top 1/10th of stocks that went up and short the bottom 1/10th), which is subject to major periods of drastic underperformance. This is all well documented in the literature. The other type is using momentum on the market as a whole, perhaps switching between asset classes based on momentum. I would not think that you could assess a strategy based on 6 months of performance.

My view as an investor since the 1980s is that the EMH is true to a 0th approximation. However massive agency issues in the fund management industry leave room to outperform on a risk adjusted basis if your incentives are different from the average fund manager. Some anomalies are just not exploitable by fund managers/agents because they would lose all their funds under management after periods > 12 months of underperformance.

LW readers interested in the topic may like reading the Alphaarchitect blog. https://alphaarchitect.com/blog/

1Richard Meadows7moSure—I was testing a dual momentum strategy over the market as a whole, with a 12-month lookback period. The 'dual' refers to both absolute momentum, and relative momentum between asset classes (bonds, US stocks, non-US stocks). I haven't evaluated it properly yet, but the signals it generated told me to stay in stocks until the end of March, at which point I ought move into bonds, just in time to miss the recovery. Over the period I've tested it so far (16 months) it has returned -4%, while my benchmark is at +18%. I am still mildly interested to see how it pans out, but I ignored the signals and am now only tracking it on paper.
4CronoDAS6moSports bookies deal with "investors" that have net biases - lots of people will, for example, bet on the home team because they're fans. I also have it on pretty good authority that people drastically underestimate the variance in soccer matches between closely matched teams; when the World Cup comes around, a simple strategy of "bet on the underdog in every match" makes money.
1gilch7moI'm not exactly sure what you mean by "defensible". It's true that some anomalies evaporate almost immediately once they are noticed, but others persist long after that--even for decades. One notable example of a long-lived inefficiency was the capping of the Euro/Franc rate [https://en.wikipedia.org/wiki/Swiss_National_Bank#Capping_of_euro_exchange_rate] at 1.20 from 2011 to 2015. This was announced publicly by the Swiss National Bank so it was available information. A price cap is a glaring market inefficiency, exploitable with a simple grid strategy. If the EMH were true, the rate should have settled at 1.20 almost instantly, but in fact, it took about three years for this to happen. And Forex is about as liquid as markets get.

Here's a thought experiment: Suppose that a market is perfectly efficient, except that every 50 years or so there's a crash, which sufficiently smart people can predict a month in advance. Would you say that this market is efficient? Technically it isn't, because smart people have a systematic advantage over the market. But practically, no trader systematically beats the market, because no trader lives long enough!

I suppose you can create a long-living institution, a "black swan fund", that very rarely makes bets on predictable crashes, and over a few centuries can prove it has higher returns. But I guess not enough people care about returns over these timescales.

FWIW, I very much wanted to adjust my investments based on the covid news, but was stopped from doing so mainly via friction:

1. I made recent profits from my stocks, so selling them (even for a short window) would have had bad capital gains repercussions for me... in the US it's a PITA to deleverage stocks that are in the green.

2. I'm not enough of a trading wonk to feel comfortable buying puts, and the little in the past that I've played around with those tools seemed to indicate that it's hard for a non-professional to avoid excessive... (read more)

There is a simple explanation to this: It turns out that being a good rationalist with access to an option trading account is actually an edge.

I also realized that the market was wrong and told friends to pull out of equities in mid-February. They didn't listen.

Curated, this is a great and really grounding contribution to the ongoing conversation around this, it explains lots of key arguments really well. I hope to see more discussion about whether the EMH is dead and what can be learned from the covid situation, and I liked much of the comments section as well.

Something that occured to me while reading this post:

In a given situation, some people who are not at the bleeding edge of finance in general will be well placed to beat the market in a way we would tend to think of as due to skill rather than luck.

In the coinflip example, the people who correctly predict the sequence do so through luck. We call the sequence random because we are unable to model the process which generates it, and we call it luck when they are able to predict it because they don't know how to model it either. In the case of detecting... (read more)

1Slider7moIn general if someone used a weird voodoo ritual it would be natural to liken that to a coinflip. If the voodoo ritual starts to be consistently profitable then it can be likened to a deliberate technique. It might not be that markets are especiallly skillfull or wise but they get to define what counts as success so automatically win in that dimension.

I think one key question, when talking about the EMH, is what we mean for an information to be available.

It's plausible, that for the public it only became clear around 27. February that Covid19 would be huge.

But it seems some experts knew much earlier. Just a quick browse on epidemiological twitter, for example, and you can find quite some instances of people expecting this not to be contained in the beginning of February. There's also the case of a swiss epidemiologist who was one of the first to warn swiss national media about Covid19 and cla... (read more)

2AllAmericanBreakfast7moMarkets did react just about as soon as COVID started making the American news. If you look at VIX, which is an index of S&P500 futures volatility (aka the "fear index"), it jumped to a new, higher plateau right as COVID-19 started making the news in mid-January.
2clearthis7moHm, all I can find are these small bumps in the end of January. [But I can't figure out how to attach screenshots here.] I also can't really see a plateau effect afterwards. An actual reaction, from a cursory view, only seems to happen on the 20. February. I'm not capable of saying whether these bumps show a market reaction or if it's largely noise. Looking at the time before, it doesn't seem like an unusual behaviour. [But I'm really not good at properly reading such charts, so I'd be interested in how you came to your conclusions.]
3AllAmericanBreakfast7moHmm, maybe you are right. I had a discussion about this in regards to another one of my posts about COVID-19 and the stock market, but now that I'm looking again I don't really see anything like what I was describing.
2AllAmericanBreakfast7moThe steelman that VIX responded to COVID-19 in January is that it went up 40% from Jan 23-27. For whatever reason, this chart [https://finance.yahoo.com/chart/%5EVIX#eyJpbnRlcnZhbCI6ImRheSIsInBlcmlvZGljaXR5IjoxLCJ0aW1lVW5pdCI6bnVsbCwiY2FuZGxlV2lkdGgiOjQuMzA5NTIzODA5NTIzODA5LCJmbGlwcGVkIjpmYWxzZSwidm9sdW1lVW5kZXJsYXkiOnRydWUsImFkaiI6dHJ1ZSwiY3Jvc3NoYWlyIjp0cnVlLCJjaGFydFR5cGUiOiJsaW5lIiwiZXh0ZW5kZWQiOmZhbHNlLCJtYXJrZXRTZXNzaW9ucyI6e30sImFnZ3JlZ2F0aW9uVHlwZSI6Im9obGMiLCJjaGFydFNjYWxlIjoibGluZWFyIiwicGFuZWxzIjp7ImNoYXJ0Ijp7InBlcmNlbnQiOjEsImRpc3BsYXkiOiJeVklYIiwiY2hhcnROYW1lIjoiY2hhcnQiLCJpbmRleCI6MCwieUF4aXMiOnsibmFtZSI6ImNoYXJ0IiwicG9zaXRpb24iOm51bGx9LCJ5YXhpc0xIUyI6W10sInlheGlzUkhTIjpbImNoYXJ0Iiwi4oCMdm9sIHVuZHLigIwiXX19LCJzZXRTcGFuIjp7Im11bHRpcGxpZXIiOjEsImJhc2UiOiJ5ZWFyIiwicGVyaW9kaWNpdHkiOnsicGVyaW9kIjoxLCJpbnRlcnZhbCI6ImRheSJ9fSwibGluZVdpZHRoIjoyLCJzdHJpcGVkQmFja2dyb3VuZCI6dHJ1ZSwiZXZlbnRzIjp0cnVlLCJjb2xvciI6IiMwMDgxZjIiLCJzdHJpcGVkQmFja2dyb3VkIjp0cnVlLCJldmVudE1hcCI6eyJjb3Jwb3JhdGUiOnsiZGl2cyI6dHJ1ZSwic3BsaXRzIjp0cnVlfSwic2lnRGV2Ijp7fX0sImN1c3RvbVJhbmdlIjpudWxsLCJzeW1ib2xzIjpbeyJzeW1ib2wiOiJeVklYIiwic3ltYm9sT2JqZWN0Ijp7InN5bWJvbCI6Il5WSVgiLCJxdW90ZVR5cGUiOiJJTkRFWCIsImV4Y2hhbmdlVGltZVpvbmUiOiJBbWVyaWNhL05ld19Zb3JrIn0sInBlcmlvZGljaXR5IjoxLCJpbnRlcnZhbCI6ImRheSIsInRpbWVVbml0IjpudWxsLCJzZXRTcGFuIjp7Im11bHRpcGxpZXIiOjEsImJhc2UiOiJ5ZWFyIiwicGVyaW9kaWNpdHkiOnsicGVyaW9kIjoxLCJpbnRlcnZhbCI6ImRheSJ9fX1dLCJzdHVkaWVzIjp7IuKAjHZvbCB1bmRy4oCMIjp7InR5cGUiOiJ2b2wgdW5kciIsImlucHV0cyI6eyJpZCI6IuKAjHZvbCB1bmRy4oCMIiwiZGlzcGxheSI6IuKAjHZvbCB1bmRy4oCMIn0sIm91dHB1dHMiOnsiVXAgVm9sdW1lIjoiIzAwYjA2MSIsIkRvd24gVm9sdW1lIjoiI2ZmMzMzYSJ9LCJwYW5lbCI6ImNoYXJ0IiwicGFyYW1ldGVycyI6eyJ3aWR0aEZhY3RvciI6MC40NSwiY2hhcnROYW1lIjoiY2hhcnQiLCJwYW5lbE5hbWUiOiJjaGFydCJ9fX19] makes it easier to see the bump I'm talking about. It's not dramatic compared with the explosion in mid-Feb, but that accords with the idea that the market was unsure about whether COVID-19 was going to be a replay of SARS or somet

Confusion: Can multiple traders profit by bringing the exact same information to market?

I want to say yes? At least if they don't individually have enough heft to fully price it in? That would explain how multiple traders are able to exploit the same anomaly, whereas hedge funds have to cap their fund size because getting too big hurts them.

3ChristianKl7moThere's a certain amount of money that can be made of an inefficiency in the market depending on the liquidity of the underlying market. There are a limited number of people who want to take the opposite site of the trade you are making. Usually, you get dimishing returns as you bid people lower till the price is at it's efficient level.

Thanks, I loved this post. Its obviously hard to distinguish luck from skill when it comes to investment returns, I don't think focusing on money and track record would be enough to convince me of skill. I think I would need to also rationally evaluate the investment strategy to determine if it can be reasonably expected to beat the market. Common strategies like "we do what everyone else does but better", or "we invest in value/growth" would probably not convince me (risk premiums, liquidity premiums, animal spirits probably affec... (read more)

Interesting post, would love to hear your opinion about my suggested alternative to EMH, I don't see anything in your post that contradicts it. But i'm having a very hard time accepting the idea that you can't beat the market using public data and infinite amount of computing power.

3Richard Meadows7moYour formulation is nifty, and intuitively makes sense to me. I am feeling too wiped right now to think about it carefully, but my off-the-cuff response is that it has something to do with the fact that 'informational edge' is a much broader category than information about the actual underlying assets. For example, a complicated day-trading algorithm is on some level a reflection of the fact that the market is missing some information. But that information looks more like 'there is a complex relationship between assets under XYZ specific conditions' than 'the EBTDA figure in Walmart's quarterly report'. I guess this is what most anomalies represent: they are more like meta-information than information. In which case, if you had a superintelligence with near-infinite compute, I think your intuition that it could indeed beat the market is right. I don't know much about quants, but I imagine this is pretty much what they are trying to do, with the difference being that they have bounded resources.
3Thomas Kwa7moIndeed, Jalex Stark [http://www.jalexstark.com/] is a quant and says [http://www.jalexstark.com/]: "I spend most of my days working on specific (proprietary) instances of the general problem "design and enact decision procedures that identify market inefficiencies as well as possible, measured in terms of maximizing the ratio (expected value in dollars of trading against the inefficiency) / (amount of human time required to find the inefficiency and execute the trades)."

If the market regularly reacts to public statements of inefficiency ("stocks systematically rise on the third Thursday of each month but only under a waxing moon") by eliminating it, then this is *refutation* of the efficient market hypothesis.

Inefficiencies are priced in when they are: obvious, easy, and safe,. A non-obvious inefficiency is a true hidden edge. A non-easy inefficiency has a high barrier (capital, technical, physical, etc) to entry. A non-safe inefficiency has a high social barrier to entry.

Because names have power, I dub this the Passive Market Hypothesis.

The EMH Gets Stronger With Every Attack

In case you weren't aware, this point has been made here before, in a classic old post: Markets are Anti-Inductive.

3Richard Meadows7moThanks, nice post. I like 'anti-inductive markets', not least because it doesn't come with all the confusing connotations of 'efficient'.

I'm probably missing something - but when you say the vast majority of active investors do really badly, shouldn't that be impossible too? If markets are truly efficient, isn't it just as hard to underperform them as outperform?

You write quite entertainingly.

There's no room for luck in the "realistic" half of your diagram.

There were some sizable selloffs starting January 22nd.

Really excellent discussion of the EMH. Love your description of it as some kind of demon-god creature.

I'll jump in on a minor point.

"And yet, and yet…even when criminals have advance access to earnings reports, they still don’t do all that well, which is evidence for the very strongest form of the EMH (the one that no-one, including me, believes can possibly be true)."

This might instead be evidence that there is a lot more insider trading than we think -- making it so that those we know of were competing for gains against other insider traders.

Your second quote that you attributed to me was actually from keaswaran, who I agree with.

1Richard Meadows7moWhoops, thanks, will fix that now.

I think you're understating the amount of private information available to anyone with a reasonable level of intelligence. If you have a decent level of curiosity, chances are that you know some things that the rest of the world hasn't "caught onto" yet. For example, most fans of Tesla probably realized that EVs are going to kill ICEs and that Telsa is at least 4 years ahead of anyone else in terms of building EVs long before the sudden rise in Tesla stock in Jan 2020. Similarly, people who nerd out about epidemics predicted the scal... (read more)

3ChristianKl7moWhen you say rest of the world it's important to keep in mind that it's not just the average person on the street you are competing with. It's professional traders. Plenty of Venture Capitalist underform the market. Saying that everyone of them beats the market is not based on real data.
1Logan Zoellner4moI didn't say every Venture Capitalist beats the market. Venture Capital in particular seems like a hobby for people who are already rich. I said every founder of a $1B startup beat the market. I propose the following bet: take any founder of a $1B startup that you please, strip them of all of their wealth, give them $1M cash. What percent of them do you think would see their net-worth grow by more than the S&P 500 over the next 10 years? If the EMH is true, the answer should be 50%. Would you really be willing to bet 50% of them will under preform the market?
3ChristianKl4moThere's a lot of survivership bias in that claim. No, you would only assume that if you bill the capacity of that founder to work at zero. Successful founders have skill at managing companies that distinct from having access to private information.
1Logan Zoellner4moCare to elucidate the difference between "skilled at managing companies" and "skilled at investing". Do you really claim that if I restricted the same set of people to buying/selling publicly tradable assets they would underperform the S&P 500?
2ChristianKl4moTennis is not about tapping private information. Management is not about tapping private information. The set of people buying/selling publicly tradable assets is the set of money managers at mutual funds. Those do underperform [https://www.cnbc.com/2019/03/15/active-fund-managers-trail-the-sp-500-for-the-ninth-year-in-a-row-in-triumph-for-indexing.html] the S&P 500. Additionally, the EDH says that all inefficiencies that can be profitably exploited disappear. If I can hire an analysts for 100k that allows me to find an efficiency that makes me 100k I'm not outperforming the S&P 500. The EDH assumes that the price you have to pay to uncover marginal inefficencies is equal to the profits that you can make from the inefficency. The most skilled analysts likely do get employed by funds, the funds still overall underperform.
1Logan Zoellner4moOh yeah, I definitely agree that mutual funds are terrible. Pretty sure they're optimizing for management fees, though, not to actually outperform the market [https://www.institutionalinvestor.com/article/b1mq53b3bnn3r3/One-of-the-Best-Market-Neutral-Funds-Is-Run-by-a-Robot] . I'm still curious if you would be willing to bet against a fund run exclusively by founders vs the S&P 500. Saying the management fee for such a fund would be ridiculously high seems like a reasonable objection though. For that matter, would you be willing to bet against SpaceX vs the S&P 500?
3marcogiglio7moPrivate information should be very hard to come by, it is not something that can be learned in a few minutes from an internet search. That Tesla is far ahead in EV technology than other car producers it is not private information, so it won't give you any advantage in trading its stock. There are many other aspects to building a successful car company than just the technological aspects, and Tesla could still very well fail there. It seems also that you implying that January 2020 rise in Tesla price is due to the market suddenly realizing the Tesla lead, but there could many alternative explanations. In fact, it is a highly shorted stock, so for sure your position is debated by many smart people. Venture capitalists do have access to private information, from financials to getting to known the founders, to internal metrics/datasets on what makes a company successful. I agree that to be a successful entrepreneur you need to tap private information/build edges. To me it doesn't look trivial/nor easy at all: there are orders of magnitude more intelligence people than rich intelligence people.
1Logan Zoellner4moI think we have different definitions of private information. I have private information if I disagree with the substantial majority of people, even if everything I know is in principle freely available. The market is trading on the consensus expectation of the future. If that consensus is wrong and I know so, I have private information. Specifically, when Tesla was trading at $600 or so, it was publicly available [https://asia.nikkei.com/Business/Automobiles/Tesla-teardown-finds-electronics-6-years-ahead-of-Toyota-and-VW2] that they were building cars in a way that no other company could, but the public consensus was not that they were therefore the most valuable car company in the world. Similarly, SpaceX is currently valued [https://www.cnbc.com/2020/07/23/spacex-is-raising-up-to-1-billion-at-44-billion-valuation.html] at $44B according to the public consensus. But I would be willing to be a substantial sum of money that they are worth 5-10x that and people just haven't fully grasped the implications of Starlink and Starship. When you think about private information this way, in order to have private information all you have to do is: 1) Disagree with the general consensus 2) Be right Incidentally, those are precisely the skills that rationality is training you for. Most people aren't optimizing for the truth, they're optimizing for fitting in with their peers. Very few intelligent people are optimizing for "make as much money as possible". A trivial example of this, almost anyone working in academia could get a massive pay raise by switching to private industry. In addition, people can be very intelligent without being rational, so even if they claim to be optimizing for wealth they might not be doing a very good job of it. There are hordes of very intelligent people who are goldbugs or young earth creationists or global warming deniers. Why should we expect these people to behave rationally when it comes to financial self-interest when they so blata

nitpick

The EMH is the only theory that grows stronger with every attack against it.

Any antifragile systems is like this

EtA: Although I guess it's extremely (perfectly?) anti-fragile.

The efficient market says that the market price reflects "all available information". That's tautological, and it's hard to argue with tautologies. The problem is that trades are made by people or algorithms designed by people. Different people have different tolerances for risk, different ideas about market performance and different strategies for making money. What does it even mean for a price to reflect all available information? A lot of that information is embedded in people's minds and situations and in our institution struc... (read more)