Markets are Anti-Inductive


30


Eliezer_Yudkowsky

I suspect there's a Pons Asinorum of probability between the bettor who thinks that you make money on horse races by betting on the horse you think will win, and the bettor who realizes that you can only make money on horse races if you find horses whose odds seem poorly calibrated relative to superior probabilistic guesses.

There is, I think, a second Pons Asinorum associated with more advanced finance, and it is the concept that markets are an anti-inductive environment.

Let's say you see me flipping a coin.  It is not necessarily a fair coin.  It's a biased coin, and you don't know the bias.  I flip the coin nine times, and the coin comes up "heads" each time.  I flip the coin a tenth time.  What is the probability that it comes up heads?

If you answered "ten-elevenths, by Laplace's Rule of Succession", you are a fine scientist in ordinary environments, but you will lose money in finance.

In finance the correct reply is, "Well... if everyone else also saw the coin coming up heads... then by now the odds are probably back to fifty-fifty."

Recently on Hacker News I saw a commenter insisting that stock prices had nowhere to go but down, because the economy was in such awful shape.  If stock prices have nowhere to go but down, and everyone knows it, then trades won't clear - remember, for every seller there must be a buyer - until prices have gone down far enough that there is once again a possibility of prices going up.

So you can see the bizarreness of someone saying, "Real estate prices have gone up by 10% a year for the last N years, and we've never seen a drop."  This treats the market like it was the mass of an electron or something.  Markets are anti-inductive.  If, historically, real estate prices have always gone up, they will keep rising until they can go down.

To get an excess return - a return that pays premium interest over the going rate for that level of riskiness - you need to know something that other market participants don't, or they will rush in and bid up whatever you're buying (or bid down whatever you're selling) until the returns match prevailing rates.

If the economy is awful and everyone knows it, no one's going to buy at a price that doesn't take into account that knowledge.

If there's an obvious possibility of prices dropping further, then the market must also believe there's a probability of prices rising to make up for it, or the trades won't clear.

This elementary point has all sorts of caveats I'm not bothering to include here, like the fact that "up" and "down" is relative to the risk-free interest rate and so on.  Nobody believes the market is really "efficient", and recent events suggest it is less efficient than previously believed, and I have a certain friend who says it's even less efficient than that... but still, the market does not leave hundred-dollar-bills on the table if everyone believes in them.

There was a time when the Dow systematically tended to drop on Friday and rise on Monday, and once this was noticed and published, the effect went away.

Past history, e.g. "real estate prices have always gone up", is not private info.

And the same also goes for more complicated regularities.  Let's say two stock prices are historically anticorrelated - the variance in their returns moves in opposite directions.  As soon as everyone believes this, hedge-fund managers will leverage up and buy both stocks.  Everyone will do this, meaning that both stocks will rise.  As the stocks rise, their returns get more expensive.  The hedge-fund managers book profits, though, because their stocks are rising.  Eventually the stock prices rise to the point they can go down.  Once they do, hedge-fund managers who got in late will have to liquidate some of their assets to cover margin calls.  This means that both stock prices will go down - at the same time, even though they were originally anticorrelated.  Other hedge funds may lose money on the same two stocks and also sell or liquidate, driving the price down further, etcetera.  The correlative structure behaves anti-inductively, because other people can observe it too.

If mortage defaults are historically uncorrelated, so that you can get an excess return on risk by buying lots of mortages and pooling them together, then people will rush in and buy lots of mortgages until (a) rates on mortgages are bid down (b) individual mortgage failure rates rise (c) mortgage failure rates become more correlated, possibly looking uncorrelated in the short-term but having more future scenarios where they all fail at once.

Whatever is believed in, stops being real.  The market is literally anti-inductive rather than anti-regular - it's the regularity that enough participants induce, which therefore goes away.

This, as I understand it, is the standard theory of "efficient markets", which should perhaps have been called "inexploitable markets" or "markets that are not easy to exploit because others are already trying to exploit them".  Should I have made a mistake thereof, let me be corrected.

Now it's not surprising, on the one hand, to see this screwed up in random internet discussions where a gold bug argues from well-known observations about the past history of gold.  (This is the equivalent of trying to make money at horse-racing by betting on the horse that you think will win - failing to cross the Pons Asinorum.)

But it is surprising is to hear histories of the financial crisis in which prestigious actors argued in crowded auditoriums that, previously, real-estate prices had always gone up, or that previously mortage defaults had been uncorrelated.  This is naive inductive reasoning of the sort that only works on falling apples and rising suns and human behavior and everything else in the universe except markets.  Shouldn't everyone have frowned and said, "But isn't the marketplace an anti-inductive environment?"

Not that this is standard terminology - but perhaps "efficient market" doesn't convey quite the same warning as "anti-inductive".  We would appear to need stronger warnings.

PS:  To clarify, the coin example is a humorous exaggeration of what the world would be like if most physical systems behaved the same way as market price movements, illustrating the point, "An exploitable pricing regularity that is easily inducted degrades into inexploitable noise."  Here the coin coming up "heads" is analogous to getting an above-market return on a publicly traded asset.