Your formulation is nifty, and intuitively makes sense to me. I am feeling too wiped right now to think about it carefully, but my off-the-cuff response is that it has something to do with the fact that 'informational edge' is a much broader category than information about the actual underlying assets.

For example, a complicated day-trading algorithm is on some level a reflection of the fact that the market is missing some information. But that information looks more like 'there is a complex relationship between assets under XYZ specific conditions' than 'the EBTDA figure in Walmart's quarterly report'. I guess this is what most anomalies represent: they are more like meta-information than information.

In which case, if you had a superintelligence with near-infinite compute, I think your intuition that it could indeed beat the market is right. I don't know much about quants, but I imagine this is pretty much what they are trying to do, with the difference being that they have bounded resources.

The EMH Aten't Dead

by Richard Meadows 18 min read15th May 202076 comments

140


Cross-posting from my personal blog, but written primarily for Less Wrong after recent discussion here.


There are whispers that the Efficient-Market Hypothesis is dead. Eliezer's faith has been shaken. Scott says EMH may have been the real victim of the coronavirus.

The EMH states that “asset prices reflect all available information”. The direct implication is that if you don’t have any non-available information, you shouldn’t expect to be able to beat the market, except by chance.

But some people were able to preempt the corona crash, without any special knowledge! Jacob mentioned selling some of his stocks before the market reacted. Wei Dai bought out-of-the-money 'put' options, and took a very handsome profit. Others shorted the market.

These people were reading the same news and reports as everyone else. They profited on the basis of public information that should have been priced in.

And so, the EMH is dead, or dying, or at the very least, has a very nasty-sounding cough.

I think that rumours of the death of efficient markets have been greatly exaggerated. It seems to me the EMH is very much alive-and-kicking, and the recent discussion often involves common misunderstandings that it might be helpful to iron out.

This necessarily involves pissing on people's parade, which is not much fun. So it's important to say upfront that although I don't know Wei Dai, he is no doubt a brilliant guy, that Jacob is my favourite blogger in the diaspora, that I would give my left testicle to have Scott's writing talent and ridiculous work ethic, that Eliezer is a legend whose work I have personally benefited from greatly, etc.

But in the spirit of the whole rationality thing, I want to gently challenge what looks more like a case of 'back-slaps for the boys' than a death knell for efficient markets.

First: how the heck did the market get the coronavirus so wrong?

The Great Coronavirus Trade

Lots of people initially underreacted to COVID-19. We are only human. But the stockmarket is not only human—it’s meant to be better than this.

Here’s Scott, in A Failure, But Not of Prediction:

The stock market is a giant coordinated attempt to predict the economy, and it reached an all-time high on February 12, suggesting that analysts expected the economy to do great over the following few months. On February 20th it fell in a way that suggested a mild inconvenience to the economy, but it didn’t really start plummeting until mid-March – the same time the media finally got a clue. These aren’t empty suits on cable TV with no skin in the game. These are the best predictive institutions we have, and they got it wrong.

But… this isn't how it went down. As AllAmericanBreakfast and others pointed out in the comments, the market started reacting in the last week of February, with news headlines directly linking the decline to the ‘coronavirus’. By the time we get to mid-March, we’re not far off the bottom.

(You can confirm this for yourself in a few seconds by looking at a chart of the relevant time period.)

EDIT: Scott has explained his rationale here. Although I still think his version of events is incorrect as phrased, I want to make it clear I am not accusing him of deliberately massaging the data or any other such shenanigans, and the next paragraph about revisionist history etc was only meant to be a general observation about how people responded. My apologies to Scott for the unclear wording, as well any perceived slight against his very good reputation.

For whatever reason, COVID-19 seems to be a magnet for revisionist history and/or wishful thinking. In other comments under the same post, the notion that people from our ‘tribe’ did especially well also comes under serious question—in fact, it looks like many of the names mentioned seem to have jumped on the bandwagon after it was obvious, and certainly long after the market was moving.

The facts are that the market reacted faster than almost all of us. But not before a few prescient people placed their bets!

So now the question becomes: why didn’t the market react earlier than February 20, like those smart people did?

The null hypothesis is that the market reacted exactly appropriately on the basis of the information available. After all, there were other potential pandemics in the recent past that were successfully contained or eradicated.

On February 20, there were only 4 known cases in Italy. We were a long ways from the bloodbath that was coming. Maybe it was correct to move cautiously until further information came in?

Here's keaswaran:

EDIT: previously misattributed to AllAmericanBreakfast (who agrees with it).

[At that time] it may have been plausible for many people to think this would continue to play out like SARS – East Asia would solve their problem, everyone else would watch airport arrivals and quarantine them effectively, and within a few weeks everything would stabilize and gradually go away. By Feb. 27 it was clear that this wasn’t happening, since community spread was very clear from the public data in Italy and Iran, and probably also clear from genetic data in the United States and elsewhere.

So we reached a tipping point in those next few days, at which point, the market started responding more vigorously.

If the null hypothesis is true, then those early trades were not quite as prescient as they look. We might be making the mistake of ‘resulting’, and confusing the reality we ended up in with all the others which were possible at the time, in which those traders lost their shirts. It's really hard to have a useful object-level discussion about this, because these events are one-offs (this is the same argument we get into every year on Scott's predictions threads!) It's not like we can run the experiment again, and thank goodness for that.

Nevertheless, Wei Dai suggested that this was the final nail in the coffin of EMH—at least for him.

I want to pause here to give mad props to Wei Dai for being totally open about everything, and especially for doing the following:

  • warning that options are dangerous, and you can easily lose the lot
  • generic disclaimer about seeking financial advice
  • updating the thread after he ended up losing 80% of his paper profits
  • mentioning selection bias, and saying we'd be right to discount his evidence

Which only leaves the initial claim that "at least for me this puts a final nail in the coffin of EMH."

This is a polite way of hinting that you might be a brilliant investing wizard with the power to beat the market. Honestly, after making such a beautiful trade—and my gosh it really was beautiful—whom amongst us could resist that temptation? Certainly not me. And anyway, it might even be true!

In making sense of claims of this nature, the first thing we have to establish: what does it even mean to be able to beat the market?

Can Uncle George Beat the Market?

Uncle George really likes his new iPhone. Man, these things are nifty! The dancing poop emoji is hilarious. On the strength of this insight, George dials his broker and loads up on AAPL stock.

the chosen one! the scourge of efficient markets! the stuff of eugene fama’s nightmares!

Over the next year, AAPL stock goes up 15 per cent, while the broader S&P 500 only goes up 10 per cent. George becomes insufferable at family dinners as he holds forth on his stock-picking powers. Guess the market isn’t so ‘efficient’ after all, huh. Suck it, Eugene Fama!

So: did Uncle George beat the market?

In the narrowest possible sense… yes.

In the sense in which we aim to string words together so that they mean things: no, of course not. By this definition, every single trade leads to one of the two parties ‘beating the market’. Millions of people beat the market while I wrote this sentence. I can flip a coin between Pepsi and Coke right now, and have a 50 per cent chance of becoming a market-beating genius.

The Uncle George example makes it glaringly obvious that a successful trade does not somehow ‘break’ efficient markets. And yet, this is the same naive criticism constantly leveled against the EMH: if the market moves in literally any direction, that must mean it was wrong before! My cousin who sold/bought before it went up/down beat the market!

Same goes for the Great Coronavirus Trade. The fact that some people got out of the market early is not even the tiniest bit surprising. Investors constantly think the market is going to crash, for any number of plausible reasons. This is the default state of affairs: we have successfully predicted 73 of the last five market crashes, etc.

These predictions are almost always wrong:

And almost all the people who make them would have been much better off taking the boring ‘buy and hold forever’ strategy.

But even a stopped clock is right twice a day. And of course, we’re much more likely to hear about the occasional brilliant successes than the near-constant dull failures.

So the most naive criticism of the EMH boils down to ‘it's possible to make a good trade’. This is just a property of trading. It tells us exactly nothing about the market’s efficiency, or lack thereof.

But some people really do beat the market—and not in the trivial sense. They're not merely stopped clocks, or highly visible 'survivors'. I’ll suggest a definition later on which strips out the effect of randomness.

Before we get there—doesn’t the concession that people can non-trivially beat the market already drive a stake through the EMH’s heart?

This is the second great misunderstanding: there is no conflict between the EMH and beating the market. That’s how the market gets efficient! You find an information asymmetry that isn’t priced in yet, and in exploiting it, you move the market a little further towards efficiency.

Let’s call this information asymmetry an ‘edge’.

If the EMH is true—or even just true-ish—that doesn’t mean the market can’t be beat. It means:

You shouldn’t expect to beat the market without a unique edge, except by chance

Now, this usually gets simplified down to ‘you can’t beat the market’. And most of the time, this simplification is good enough: you might get lucky and win in the Uncle George sense, but over an investing lifetime, you’ll almost certainly revert to the mean (which isn’t matching the market return—it’s underperforming it).

But if you can find some kind of edge, you really can win! So, what might a genuine edge look like?

Anomalies Exist!

The Uncle Georges of the world don’t have an edge. All of their thoughts have already been thunk by someone else (probably by millions of someone elses). Instead, their fortunes are entirely at the mercy of the myriad other forces that drive stock prices: consumer demand, workplace harassment scandals, money printers going brrr, the exact virulence of a novel coronavirus, the price of cheese in Spain last Friday afternoon, etc.

All of this stuff—billions of inputs processed by the greatest collective intelligence ever built—is a black box unto us mere mortals. It’s impossible to assign perfect causal explanations to stock prices, which means we can pick whichever story suits us best. As a market reporter, this was pretty much my whole job: calling brokers and economists to wrap a plausible narrative around totally inexplicable events, and generate sage nodding of heads.

All Uncle George can see is that he placed his bet, and AAPL went up. It was the poop emoji for sure!

And so, Uncle George spends his days dishing out hot stock tips on online forums, oblivious to the fact that his success was meaningless.

[uncle georging intensifies]

What does a real edge look like?

A century ago, investors started noticing they could consistently pick up bargains by running very simple formulas over stock prices. The most famous is the ‘value investing’ approach developed by Ben Graham, and used by Warren Buffett and Charlie Munger. There was a genuine, big old inefficiency in the markets, and these guys had a great time exploiting it.

I think this might be the image most people have in their head when they think of ‘beating the market’—diligently studying The Intelligent Investor and learning about PE ratios or whatever.

But this is like trying to use a stone-age axe against a fighter jet. The Ben Graham information asymmetry has long since disappeared because…markets are efficient(ish)! Once the formula was widely known, it stopped working. Investors developed more sophisticated versions, more formulas, more pricing models. Once those got out, they stopped working too. Now there’s a great debate as to whether even the most complicated descendants of value might be totally dead. In which case, the anomaly has officially gone for good.

Either way, this is not how Buffett gets his edge, and it hasn’t been for decades. Here’s Munger:

The trouble with what I call the classic Ben Graham concept is that gradually the world wised up and those real obvious bargains disappeared. You could run your Geiger counter over the rubble and it wouldn’t click.

Buffett’s most brilliant achievement is weaving this folksy legend that he is a cute old grandpa who beats the market by backing the best companies. Let's take a look at how market-beating investors really make their money.

Modern Edges are Completely Ridiculous

greatest showman on earth

1. The Warren Buffett Halo Effect

In recent decades, Buffett has made a killing through juicy private deals which are completely out of reach of the average investor. Like, six billion dollar deals with three billion in preference rights and a guaranteed dividend. Like, lobbying the government to bail out the banks, then carving off a huge piece of the action. Like, being able to play around with Berkshire Hathaway’s $115 billion insurance float. Much of his fortune is built on taxpayer largesse.

Warren Buffett’s brand is so powerful that at this point, his success is a self-fulfilling prophecy: when Berkshire invests in a stock, everyone else piles in after him and drive the price up. Buffett even lends out his ‘halo’ to companies that need it—most famously during the GFC—so long as they give him a generous discount to the market price, of course. (Matt Levine has written some fascinating columns about this).

And yet, and yet… Berkshire Hathaway has underperformed for the last decade. Buffett would have been better off to take his own advice and put it all in index funds.

2. Hedge funds with armies of drones

There you are, sitting in your home office going through Walmart’s quarterly report and calculating PE ratios or whatever. Meanwhile, the professionals are using an army of drones to monitor the movement of shopping carts in Walmart parking lots in real time.

See also: sending foot soldiers out to every branch of a bakery chain at the close of business each day, because the numbered dockets start out at zero, and thus contain live sales data unavailable to the market.

And so, when renowned hedge fund manager Michael Steinhardt was asked the most important thing average investors could learn from him, here’s what he suggested:

“I’m their competition.”

And yet, and yet…almost all hedge funds underperform. Not all of them are trying to beat the market, but the tools at their disposal gives us a sense of the difficulty here.

3. High-frequency traders move mountains

If multiple people have access to the same information, the speed in which you can bring it to market also matters. So, we have high-frequency traders.

One firm spent $300 million laying a direct cable between Chicago to New Jersey. They cut straight through mountains and crossed rivers. The cable stretched 1331 kilometres. And they did this to shave four milliseconds off their transmission time.

And yet, and yet…microwaves came along and rendered the whole project obsolete. Trying to get an edge is expensive.

4. Being willing and able to commit felonies

Insider trading is a thing. See also: criminals who hack or otherwise steal sensitive private information.

And yet, and yet…even when criminals have advance access to earnings reports, they still don’t do all that well, which is evidence for the very strongest form of the EMH (the one that no-one, including me, believes can possibly be true).

So…what sort of edge do us lesser mortals have?

If we mumble something about having ‘good intuition’, or ‘subscribing to the Wall Street Journal' then we should consider the strong possibility that we are Uncle George.

If the answer involves ‘fundamental analysis’ or ‘Fibonacci retracements’, we're still in Uncle George territory. The only difference is that doing something complicated makes it easier to internally justify the belief that we know a secret no-one else does. But it is still (probably!) a mistaken belief.

The EMH Gets Stronger With Every Attack

So we know for sure that market-beating edges exist—I’ve even written them down for everyone to see!

I can only dream of possessing a halo effect so strong that everyone piles into a stock right after I announce I have graced it with my favour. I don’t have an army of drones at my command, or the ability to bore through mountains to shave milliseconds off my trading times, or a weekly round of golf with the CEO of a Fortune 500 company.

The market is never perfectly efficient. But relative to me, it might as well be.

Critics have pointed out plenty of cases in which the EMH doesn’t jive with reality—and they are absolutely right. So this is where it gets really weird.

The EMH is the only theory that grows stronger with every attack against it.

Every edge is constantly at risk of being gobbled up by an efficient-ish market. The ones I’ve mentioned can only be publicly knowledge because they're somewhat defensible: they’re based on personal relationships, capital investment, proprietary technology, etc. But they disappear too.

Most edges can’t even be spoken out loud without disappearing. If stocks systematically rise on the third Thursday of each month but only under a waxing moon, and then someone writes about it in public, you can kiss that anomaly goodbye. The EMH sucks it into its gigantic heaving maw, and it’s gone forever.

In other words: every time someone picks a hole in the theory and points out an inefficiency, they make the predictions generated by the EMH more robust! It’s like some freaky shoggoth thing that Just. Won’t. Die.

you may not like it, but this is what peak efficiency looks like

Which gets us to the totally justified criticism of the theory: the only reason the EMH can pull this stunt is because it’s bullshit science.

It’s unfalsifiable! It responds to criticism by saying, ‘OK, good point, but now that I’ve factored that in, you should believe in my theory even more.’

And…we really should?

The only way I can think about the EMH without going insane is to remind myself that it generates a useful heuristic. It’s not a stable law, like we might find in hard sciences. It’s not perfectly accurate. At any given point in time, there are always competing models that do a better job of describing reality. But all those other models can stop working at any moment, with no warning! By the time you find out their predictive power is gone, it’s too late, and you probably lost a bunch of money! By contrast, the EMH is a reliable model—reliably vague and hand-wavy, yes, but also reliably useful.

We know there are inefficiencies in the market. In the fullness of time, they will be absorbed into the gelatinous alien-god’s hivemind. But before that happens, maybe we can make some money off of them.

So now we come to the final test. How do you tell if you’ve really found a market-beating edge—that is, a model of reality that has more predictive power than the EMH—or you’re fooling yourself like Uncle George?

If You’re so Smart, Why Aren’t You Rich?

Everyone knows a secret about themselves, or the people they know well, or can arbitrage some opportunity in a niche that few people are paying attention to. These illiquid private ‘markets’ are much more fertile hunting grounds for asymmetries, and something I encourage everyone to think about.

But the public security markets are a gigantic agglomeration of everyone’s predictions, which constantly hoovers up every new fragment of information, and recalibrates itself in real time. Your challenge is to try and predict why this giant meta-prediction is wrong, and in which direction.

If you think you can reliably beat, say, an index fund that passively tracks the S&P 500, this is a much stronger claim than it first appears.

For one thing, you’re claiming to be better than Warren Buffett, who has failed to pull this off in the last 10 years, despite his huge advantages, and has started saying the game is so hard that everyone should just buy index funds. But that’s nothing. You are also claiming that you have the power to beat the greatest collective intelligence humanity has ever created.

This is an extraordinary claim, and the thing about extraordinary claims is that they require extraordinary evidence.

Uncle George’s AAPL trade ain’t going to cut it. Here is the extraordinary evidence that I would personally want to see before agreeing that an investor can beat the market:

1. Big heaps of money

This is the one area of life where there really is no dodging that most venerable of sick burns: if you’re so smart, why aren’t you rich?

So the first piece of evidence I would accept is the fact that someone is very, very rich. Sitting atop a big old pile of cash. And they’d probably also open a hedge fund so they can take other people’s money and turn it into millions more.

2. Track record of outperformance

Maybe a genuine market-beater doesn’t have enough starting capital to make big piles of money on a relatively slim edge, and for some reason is unable to come up with any scheme to beg or borrow more? Or the anomaly is real, but disappears before they can get filthy rich?

In these scenarios, I would also accept a complete record of out-of-sample investment returns over time—no backtests! no selecting the best trades!—as compared against the appropriate risk-adjusted benchmark.

These evidential standards work both in the case that the EMH is ‘dead’, i.e. you can reliably beat the market using public domain info which ought to already be priced in, and in the case that it’s not, i.e. you really do need novel information to get an edge.

As for evidence that the EMH really is dead…hmm. It’s not a proper theory to begin with. But I guess it would be ‘dead’ when the predictions it generates stop being accurate or useful? Which would look something like: ‘finding out that plenty of people meet the standards above, despite having never been in possession of a scrap of information that wasn’t already available to the market’.

In doing so, we’d have to be very careful to make sure we aren’t just looking at the Uncle Georges who unwittingly drew the winning lottery ticket. After all, we should expect plenty of investors to beat the market for a long time—even for years on end—entirely by chance.

The Great Coin-Tossing Experiment

Say we held a national coin-flipping contest. After 15 rounds, one in every ~32,800 people would have managed to call every single toss correctly, perfectly predicting a sequence like this:

H T H H T T H H H H T H T T T

Pretty impressive, huh!

Well, only in a world where we don’t know about probability. In that world, we might mistake blind randomness for skill. The lucky few winners would be hailed as the heroes of their hometowns, do interviews with breathless breakfast TV hosts, and explain that it’s all about the precise flick of the wrist. Aspiring flippers would queue up to buy the inevitable best-selling book, Flip Me Off, and pay exorbitant sums for one-on-one coaching sessions with the master tossers.

Depressingly, this is exactly what happens in the world of investing. What does it mean to achieve the kind of success which only happens by chance with 0.0003 probability? In the United States alone, it means you end up with 10,000 lucky dopes who are indistinguishable from brilliant investors.

And fund managers don’t need to do anywhere near that well to attract a market-beating aura. They’re incentivised to swing for the fences, increasing the odds they beat the market in some highly visible fashion over some shorter period—say, a lucky season or two. They inevitably regress to the mean, sometimes crashing and burning in spectacular fashion, but it doesn’t matter so long as they manage to hose naive investors in the meantime.

We can never entirely rule out the effect of randomness—there will always be some tiny chance that Warren Buffett is really just the world’s greatest coin-flipper—but we have to draw the line somewhere, or the standard is impossibly high.

Once the odds of a fluke get pretty slim—someone is super duper rich, and they’ve made a ton of consistently good trades over time—I’d happily congratulate them on their market-beating prowess, give them all my money to invest, listen eagerly to their advice, etc.

Being good Bayesians, this is obviously a spectrum: if they are not at that point, but trending in the right direction, I would be less skeptical, etc. But the bar has to be pretty high, or there's no way to separate skill from randomness.

Scott and Eliezer have both alluded to their comments being informed by private information. Here's a reddit comment in which Scott responds to criticism of his 'EMH is the real victim' riff:

I think we're in an asymmetric position, in that I know these people pretty well, I know they've thought about efficient market before, and they're the sort of people I would expect to beat the market if anyone could. I agree that if you just hear some blogger say he saw some people beat the market once, that's not much evidence.

Eliezer definitely understands the EMH, because the descriptions of it in Inadequate Equilibria are among the most brilliant and insightful I've ever come across. And Scott is obviously super smart.

I would love to know what their evidential standards are, but I'm explicitly not calling them out, or any of the people mentioned earlier in the post. No-one is under the slightest obligation to share private evidence, and I would be thrilled if those folks were indeed the market-beating chosen ones.

But I am saying that people, in general, make these kind of claims all the time—in good faith and with no malicious intent—and in general, taking their advice is an extraordinarily bad idea.

A heuristic: if you (or someone you know) is confident they can beat the market, and yet you notice you are not sitting atop an enormous pile of wealth, it's at least worth considering the possibility that you might be fooling yourselves.

The Four Types of Investors

There are very obvious and well-known reasons why everyone loves to think they can beat the market: overconfidence, confirmation bias, ‘resulting’, selective memory, survivorship bias, etc.

These forces are so powerful that many people—myself included—blithely ignore the vast piles of evidence that suggest beating the market is incredibly difficult, and go ahead and try anyway. All of us think we are special, and (almost) all of us are wrong.

Even if we don't personally harbour this particular fantasy, there's also a natural tendency to want our tribe or our friends to be the brilliant visionaries who were ahead of the action, possess sweet market-beating skills, etc.

So we can roughly place investors into one of the following four quadrants:

('Losers' and 'winners' here is tongue-in-cheek, and not a value judgment: literally, losing/winning this game by either successfully beating the market, or failing to do so)

Deluded losers (‘Suckers’)

“Apple stock really is undervalued, but the market hasn’t recognised it yet. I just got unlucky—it was because of [elaborate rationalization]. Also, even if I got it wrong this time, I was really close. Next time!

“What’s that? Do I track my portfolio returns over time, and compare against the relevant risk-adjusted benchmark to see whether I’m actually outperforming? Well, there’s no need. I usually do pretty well for myself, and I’m expecting to improve—in fact, I just picked up this classic book called The Intelligent Investor…

Deluded winners (‘Dumb Luck’)

“I knew Apple stock was undervalued! And I remember that other time I made a really good trade, too. Guess I’m pretty good at this game!

“…What’s that? I might have just got lucky? Hah, no. I even did the Fibonacci retracements and everything."

Realistic losers (‘Clear-Eyed Fools’)

“I keep a meticulous record of my portfolio returns, which forces me to acknowledge the fact that even though I occasionally do well, I am underperforming my benchmarks on a risk-adjusted basis. I am under no delusions about my prospects of finding an edge, and I know I really ought to take Warren Buffett’s advice and put all my money in index funds.

“But I enjoy playing the markets! It's like how a night in Vegas can have negative EV but still be positive utility, because of all the non-financial factors. So I’m gonna keep gambling with a small part of my portfolio, just for shits and giggles. In the event that I ‘win’, I will try really hard to resist the incredible internal pressure to start thinking of myself as a brilliant investing guru."

Realistic winners (‘Chosen One’)

“I keep a meticulous record of my portfolio returns, which have outperformed the appropriate risk-adjusted benchmarks to such a degree that I am confident I have found a genuine informational asymmetry. I will of course never tell anyone about it, or it will become useless.

“And I can never be entirely sure: it’s also possible that I just got lucky. But at the very least, I am sitting atop great piles of money, which is pretty nice.”

***

The vast majority of people who actively trade their account are ‘Suckers’. Some smaller number fall into the ‘Dumb Luck’ quadrant (Uncle George would stay there if he never places another trade, but he almost certainly won’t be able to help himself.)

The right-hand quadrants are much more sparsely populated. I guess there are a few ‘Clear-Eyed Losers’ floating around, and a tiny handful of ‘Chosen Ones’.

This rough distribution is probably not too controversial. The question is, which one are you?

(I made a poll on my website at this point in the post, just for a bit of fun: the results so far are brilliant)

Trying to Beat the Market is Like Crack for Smart People

There is a tendency for smart people to wander into areas they know very little about, and think they can do better than the actual experts who have years or decades of domain-specific knowledge, on the basis of being very smart, or having read some blog posts online about being more rational.

This would be OK if it was just a bit cringe. I love armchair pontificating as much as the next guy! The consequences are usually limited to mildly annoying the people who actually know what they’re talking about, and much eye-rolling when you triumphantly reinvent the wheel.

There is some upside too: reinventing the wheel is fun, because you get to, like, invent wheels. And very occasionally, it might even be true! No doubt smart outsiders are occasionally able to breeze into a new field and exploit some obvious inefficiencies.

But…oh boy. It’s really not true of this particular domain. And it’s not harmless either.

The central prediction generated by the EMH is that you should not expect to be able to beat the market (in the non-trivial sense) unless you have unique information or some similar edge.

This prediction is tested every day. We have great piles of evidence which suggest that it is correct: the vast majority of active investors do really badly.

Crucially, it’s not only regular schmucks who underperform. So do paid professionals, and active managers, and hedge funds, and all sorts of brilliant people who have made this their life’s work.

It's possible that I am not making many friends with this post. I certainly feel pretty nervous about publishing it. Everyone who thinks they can beat the market will have their hackles up! If it helps at all, I am not claiming the high ground. I have made every dumb investing mistake you could think of, and then a few more besides. I am painfully aware of how hard this is.

These days I would put myself in the ‘Clear-Eyed Fool’ quadrant, but only by a fingernail. It’s a constant battle even to stay there. I still do clever things that contradict my own boring advice, and annoyingly, am rewarded for my hubris just often enough to start entertaining the thought that I’m a brilliant investing genius after all. Then I force myself to calculate the IRR on my publicly-traded investments, and compare it against appropriate benchmarks, and manage to get a fingernail-hold back on boring old reality.

To the extent that I have succeeded as an investor, and I am doing quite nicely thank you, it has only come through forcing myself to acknowledge the main prediction that emerges from the very-much-alive-and-kicking EMH. The huge and underappreciated benefit of doing so is that I occasionally divert some of my attention elsewhere, to domains where I actually do have an edge—and then I win.


Discussion

The EMH is a weird, counterintuitive, freaky-ass shoggoth of a thing and it still confuses the heck out of me, even after almost a decade of writing about finance. I almost certainly made some mistakes in the post—in the process of writing it, I noticed several ways in which my initial beliefs were subtly wrong, which has already been super useful for me, and helped me understand some of the (valid) objections people have raised.

So I would also like to use this post to open up the floor to any and all EMH discussion, and try to benefit from the smaller-but-still-powerful collective intelligence that is Less Wrong.

I'm going to add comments with some of my own questions and uncertainty. It would be nice to become less confused together, and try to get a better sense of where we should apply our efforts at the margin.

140