I think I've uncovered an error in Eliezer Yudkowsky's book Inadequate Equilibria that undermines a key point in the book. Here are some of my observations.
First, let me provide some context. In the first chapter, Yudkowsky states that prior to Shinzo Abe's tenure as Prime Minister of Japan, the Bank of Japan had implemented a bad monetary policy that cost Japan trillions of dollars in real economic growth.
His point was that he was able to spot this mistake, and confidently know better than the experts employed at the Bank of Japan, despite not being an expert in economic policy himself. In a dialogue, he wrote,
CONVENTIONAL CYNICAL ECONOMIST: So, Eliezer, you think you know better than the Bank of Japan and many other central banks around the world, do you?
ELIEZER: Yep. Or rather, by reading econblogs, I believe myself to have identified which econbloggers know better, like Scott Sumner.
C.C.E.: Even though literally trillions of dollars of real value are at stake?
To demonstrate that he was correct on this issue, Yudkowsky said the following,
When we critique a government, we don’t usually get to see what would actually happen if the government took our advice. But in this one case, less than a month after my exchange with John, the Bank of Japan—under the new leadership of Haruhiko Kuroda, and under unprecedented pressure from recently elected Prime Minister Shinzo Abe, who included monetary policy in his campaign platform—embarked on an attempt to print huge amounts of money, with a stated goal of doubling the Japanese money supply.5
Immediately after, Japan experienced real GDP growth of 2.3%, where the previous trend was for falling RGDP. Their economy was operating that far under capacity due to lack of money.6
However, that last part is not correct, as far as I can tell.
According to official government data, Japan's RGDP had not been falling prior to 2013, other than the fall caused by the Great Recession. RGDP did grow by ~2.0% in 2013, but I cannot discern any significant change in the trend after Haruhiko Kuroda began serving as governor at the Bank of Japan.
In his footnote, Yudkowsky cites this article from 2017 to provide a "more recent update" about Japan's successful monetary policy. However, I don't think the article demonstrates that Yudkowsky was correct in any major way about the point he made.
The article never presents data on RGDP. Instead, it focuses primarily on how unemployment has fallen since 2013. However, it's hard for me to see any significant impact that Japan's shift in monetary policy had on unemployment when examining the data.
The only data series presented in the article is this plot of the prime age labor force participation rate. However, the effect looks kind of weak to me, and I don't think raising prime age LFPR is a standard target of monetary policy. (For example, the Bank of Japan had a webpage called "Outline of Monetary Policy" from the time the article was published, and it focused solely on the goal of price stability at 2% inflation, and didn't mention employment levels.)
After looking at a more standard target, it seems that Japan's new monetary policy isn't achieving its goals, as Japan experienced no substantial sustained inflation after Haruhiko Kuroda took charge of the Bank of Japan in March 2013, despite their target of 2% inflation (at least until 2022).
Note that the brief spike in Japan's CPI in April 2014 was almost certainly a result of their VAT hike, rather than any change in monetary policy at the time.
That's not to say that I think the Bank of Japan was wrong to print more money. In fact, I am not aware of any strong disagreements that I have with Scott Sumner's general view on monetary policy, which is where Yudkowsky says he got (at least some of) his opinions from.
However, I think this error undermines a significant part of Yudkowsky's thesis. This example was one of two major anecdotes that Yudkowsky presented to show that he can often know better than experts, and he cited it repeatedly throughout the book. Yet, I think he got it wrong.
Just to say it explicitly, I also don't see at all how the graphs you've shown could be interpreted to support Eliezer's interpretations. I also don't understand how Eliezer arrived at the decision to tell this story in this way based on this data.
My only background on this issue is reading Inadequate Equilibria and this post, so that's all I'm going on here.
It does seem like a straightforward conclusion that Eliezer didn't really understand what he was writing about then.
If so, publishing a revised version with the necessary changes seems like the sensible choice. Especially since the example is frequently referenced throughout.
Scott Sumner offers some comments here FWIW, copying and pasting:
I'm happy that Scott Sumner commented. I think his analysis is reasonable, and I roughly agree with what he said. My only major complaint is that I think he might have misread the extent to which my article was intended as a criticism of his policy recommendations as opposed to Eliezer's specific commentary. I think it's plausible that the new monetary policy had a modest but positive counterfactual impact on RGDP over several years. I just don't think that's the impression Eliezer gave in the book when he provided the example.
It does seem plausible that the Bank of Japan thing was an error. However, I don't think that would undermine his thesis.
It's been some time since I read the book so I could be misremembering, but I remember the big idea here to basically be "consider the incentives". If there are strong incentives to be correct about something (like the price of a stock) you should be weary of disagreeing. On the other hand, if there aren't strong incentives to be correct about something, you shouldn't be too hesitant to disagree. More generally, your hesitance should be in proportion to the strength of the incentives.
I think the Bank of Japan example was meant to be illustrative of this idea, not strong evidence for it. As I understand it, the bulk of the evidence is more gears level/theoretical/deductive. Maybe something like this:
I agree that this error does not substantially undermine the entire book, much less prove its central thesis false. I still broadly agree with most of the main claims of the book, as I understand them.
Which thesis were you referring to?
I think the claims of the book along the lines of the following quote were definitely undermined in light of this factual error,
In particular, I think this error highlights that even sophisticated observers can make basic errors in reasoning that demonstrate that they don't understand a situation well.
In general, I think it's correct for non-expert observers to be skeptical that they can identify the correct blogger in a complex debate about a topic they don't fully understand. Moreover, even if sophisticated observers can identify the correct blogger, that doesn't mean that they're necessarily correct about their interpretation of what that the blogger is saying. Both of these points are important.
I see, thanks for clarifying.
Here is how I am thinking about it. Consider the claim "the Bank of Japan is being way too tight with its monetary policy". Consider two reasons why that claim might be wrong:
My read is that Eliezer was only making points about 1, not 2.
It sounds like you are saying that he was making claims about 2. Something like, "don't be too hesitant to trust your reasoning about complex topics". But even if he was making this claim about 2, I still don't think that the Bank of Japan example matters much. It would still just be illustrative, not strong evidence. And as something that is merely illustrative, if it turned out to be wrong it wouldn't be reason to change one's mind about the original claim.
No, I think he was also wrong about the Bank of Japan's relative competence. I didn't argue this next point directly in the post because it would have been harder to argue than the other points I made, but I think Eliezer is just straight up wrong that the Bank of Japan was pursuing a policy prior to 2013 that made Japan forgo trillions of dollars in lost economic growth.
To be clear, I don't think that the Bank of Japan was following the optimal monetary policy by any means, and I currently think Scott Sumner is right to think they should have printed more money. But Eliezer didn't just say that the Bank of Japan could switch to a slightly better policy on the margin. He repeated multiple times that it was a major policy failure, that cost trillions of dollars in real value to Japan, and even went on to repeat that claim but in the context of Europe. In my opinion that's a major difference. I think he's simply wrong about the cost of the policy, even if he was right about which blogger has the correct theory.
It matters what examples you use to illustrate an argument. Presumably Eliezer tried to remember times in his life in which he was able to know better than a bunch of experts, and this example came to mind as especially salient (hence why he used it as his first example). The fact that he turned out to be mistaken about the example provides significant evidence about how often he's able to know better than experts about their domain of expertise.
Hm, I think I'm still confused about what thesis you're pointing to and where, if anywhere, you and I disagree. I think we agree:
I think Eliezer as well as most reasonable people would agree with the above as well. The difficulty comes when you starting considering a specific example and getting concrete. How smart do we think the people who run the Bank of Japan are? Are they incentivized to do what is best for the country or are other incentives driving their policy? How complex is the topic?
For the Bank of Japan example, it sounds like you (as well as most others) think that Eliezer was too confident. Personally I am pretty agnostic about that point and don't have much of an opinion.
I just don't see why it matters. Regardless of whether Eliezer himself is prone to being overconfident, points 1 through 5 still stand, right? Or do you think one/part of Eliezer's thesis goes beyond them?
Why is blogging being sold as something superior to boring old books and lectures? Blogs have the advantage of speed , right enough, but the BoJ thing has dragged on for decades. And, as you say, if you try to read blogs without having the boring groundwork in place, you might not even understand them..
I don't think that it is. In this particular case Eliezer was saying he was swayed by some bloggers but not that blogging is superior in general.
It's not clear that he was correctly swayed, and it's not clear that his favourite bloggers are reliable access the board.
Even if he had been correctly swayed, it's still not clear that his favourite bloggers are reliable access the board.
The idea that there is a reliably correct contratianism remains unproven.
I'm not buying the simple distinction between correct and incorrect. The price of a stock is just an aggregate of everyone else's opinion. If that aggregate opinion was always correct, market crashes wouldn't occur.
Attempted paraphrase of this post:
At time1, Eliezer thought that Sumner's macroeconomic analysis was correct, and that it showed that the Bank of Japan's monetary policy was too tight, at a cost of trillions of dollars.
At time2, Eliezer wrote Inadequate Equilibria where he used this view of time1 Eliezer as one of his central examples, and claimed that events since then had provided strong evidence that it was true: Japan had since loosened its monetary policy, and their economy had improved.
Now, at time3, you are looking back at Japan's economy and saying that it didn't actually do especially well at that time, and also that [EDITED] its monetary policy never actually became loose enough to generate inflation. So Eliezer at time2, who thought that the views of time1 Eliezer had been tested and shown correct, was wrong about that.
Eliezer's views at time1 might have been correct, but we never got a clear empirical test of that, and Eliezer at time2 was wrong to claim that we had.
FWIW this reads as somewhat misleading to me, mainly because it seems to focus too much on "was Eliezer right about the policy being bad?" and not enough on "was Eliezer's central claim about this policy correct?".
On my reading of Inadequate Equilibria, Eliezer was making a pretty strong claim, that he was able to identify a bad policy that, when replaced with a better one, fixed a trillion-dollar problem. What gave the anecdote weight wasn't just that Eliezer was right about something outside his field of expertise, it's that a policy had been implemented in the real world that had a huge cost and was easy to identify as such. But looking at the data, it seems that the better policy did not have the advertised effect, so the claim that trillions of dollars were left on the table is not well-supported.
Parts of your description sound misleading to me, which probably just means that we have a disagreement?
My read is that, if this post's analysis of Japan's economy is right, then Eliezer's time1 view that the Bank of Japan was getting it wrong by trillions of dollars was never tested. The Bank of Japan never carried out the policies that Eliezer favored, so the question about whether those policies would help as much as Eliezer thought they would is still just about a hypothetical world which we can only guess at. That makes the main argument in Inadequate Equilibria weaker because one of its central examples of being able to see that experts were getting it wrong is untested rather than confirmed by evidence. And the book has the further flaw that the author mistakenly marked his conditional prediction as "True" rather than "N/A, antecedent not met".
Phrases like "the better policy did not have the advertised effect" and "the claim that trillions of dollars were left on the table is not well-supported" would be appropriate in a world where the Bank of Japan did raise inflation and keep it high for a few years, and real growth didn't materialize. But we're instead in a world where inflation didn't increase much.
(This is all just accepting this post's analysis of Japan's economy, for simplicity.)
I regard this claim as unproven. I think it's clear the Bank of Japan (BOJ) began a new monetary policy in 2013 to greatly increase the money supply, with the intended effect to spur significant inflation. What's unclear to me is whether this policy matched the exact prescription that Eliezer would have suggested; it seems plausible that he would say the BOJ didn't go far enough. "They didn't go far enough" seems a bit different than "they never tested my theory" though.
They did in fact not go far enough. Japanese GNI per capita growth from 2013 to 2021 was 1.02%. The prescription would be something like 4%.
I'll just note that the NGDP growth from 2013 to 2017 (when Inadequate Equilibria was published) was about 2% per year whereas RGDP went up by about 1% per year. This definitely makes me sympathetic to "they didn't go far enough" but I'm still not sympathetic to "they never tested my theory" since you'd still expect some noticeable large effects from the new policy if the old monetary policy was responsible for a multi-trillion real-dollar problem.
Trillions of dollars in lost economic growth just seems like hyperbole. There’s some lost growth from stickiness and unemployment but of course the costs aren’t trillions of dollars.
I was assuming that the lack of inflation meant that they didn't fully carry out what he had in mind. Maybe something that Eliezer, or Scott Sumner, has written would help clarify things.
It looks like Japan did loosen their monetary policy some, which could give evidence on whether or not the theory was right. But I think that would require a more in-depth analysis than what's in this post. I don't read the graphs as showing 'clearly nothing changed after Abe & Kuroda', just that there wasn't the kind of huge improvement that hits you in the face when you look at a graph, which is what I would've expected from fixing a trillions-dollar mistake. If we're looking for smaller effects, I'd want a more careful analysis rather than squinting at graphs. (And when I do squint at these graphs, I see some possible positive signs. 2013-19 real GDP growth seems better than I would've predicted if I had only seen the pre-Kuroda graph, and Kuroda's first ~year is one of the better years.)
I have one nitpick with your summary.
I'm not actually sure whether Japan's monetary policy became substantially looser after 2013, nor did I claim that this did not occur. I didn't look into this question deeply, mostly because when I started looking into it I quickly realized that it might take a lot of work to analyze thoroughly, and it didn't seem like an essential thesis to prove either way.
It didn't become loose enough to generate meaningful inflation, right? And I thought Sumner & Eliezer's views were that monetary policy needed to be loose enough to generate inflation in order to do much good for the economy.
That's what I had in mind by not "all that loose"; I could swap in alternate phrasing if that content seems accurate.
Yes, monetary policy didn't become loose enough to create meaningful inflation. That doesn't by itself imply that monetary policy didn't become loose, because the theory of inflation here (monetarism) could be wrong. Nonetheless, I think your summary is only slightly misleading.
You could swap in an alternative phrasing that clarifies that I merely demonstrated that the rate of inflation was low, and then the summary would seem adequate to me.
These graphs seem concerning to me, but I'm worried about an information cascade before Eliezer's responded or someone with substantial expertise in macroeconomic policy has weighed in, so I'm planning to refrain from voting on this post until a week from now.
(Posting as a comment in case others feel inclined to adopt a similar policy.)
Edit: I've now upvoted, since no contrary info has come in that I've seen and at least one person with experience in economics has commented supportively.
Further illustrating Eliezer's misplaced confidence, Sumner's view is about NGDP targeting, so the success of the BOJ's policy should be based on delivering NGDP growth, not real economic variables like RGDP growth or employment rate as Eliezer implies. They were in fact successful at this (RGDP growth + Inflation = NGDP growth; with RGDP growth continuing on trend and Inflation bucking the downtrend, that's a new NGDP trajectory, baby!). Here, with 100=March 2013 as Kuroda ascended, you can see the shift in CPI trend even before the VAT impact in April 2014. Sumner was bullish on the new BOJ policy by September 2013.
So, Eliezer, you think you have identified which econbloggers, like Scott Sumner, know better than the Bank of Japan, do you? Eliezer did identify Sumner successfully, but he got lucky. His belief in Sumner was based on a misread of Sumner's position, one that led him to wrongly believe real economic variables would supply evidence for the veracity of the theory. Further compounding the issue, while employment rate might have been readable as supportive, as Matthew Barnett points out, RGDP was not. He is overconfident and should be more humble about his approach.
Ironically, Eliezer's mistake actually more strongly makes his key point. The demand for humility Eliezer was writing about stemmed from the belief that even a very good reasoner oughtn't be able to outperform "the experts." And yet, here we have a mistaken reasoner outperforming "the experts" (at least, outperforming the hawkish experts, before they were replaced by the dovish experts who implemented the new monetary policy at the BOJ). Perhaps the case for humility is not so strong after all: "it is perfectly plausible for an econblogger to write up a good analysis of what the Bank of Japan is doing wrong, and for a sophisticated reader to reasonably agree that the analysis seems decisive, without a deep agonizing episode of Dunning-Kruger-inspired self-doubt playing any important role in the analysis." I suppose one might need to decide how interchangeable "humility" and "agonizing self-doubt" are...
Eliezer is driving an intellectual racecar when many are driving intellectual horse-and-buggies. Still needs to be vacuumed out from time to time though.
One person outperforming the experts once, is not very significant..it's all about reliability.
At some point in my life I was convinced by a theory of macroeconomics called modern monetary theory. I was convinced everybody was wrong about macroeconomics and I was right.
The reader might find it amusing to learn that, in stark contrast to many other issues of this type, this particular belief counterfactually cost me tens of millions of dollars.
Humbled by age, experience and debate with actual economists, no longer possessed by a Yudkowskian sense that I am right and everybody is wrong I am no longer certain I know the answer on macroeconomics. But I will point out that it is a characteristic prediction of MMT that monetary operations will have NO effect on inflation - only fiscal deficits determine inflation.
What does that mean? If you hadn't believed it, you would be tens of millions of dollars richer now?
Perhaps explain your story in more detail. Others might find it interesting.
What was the other anecdote? Was it similarly overstated?
Same as DirectedEvolution, my only background is reading inadequate equilibria and this article. Does someone, after reading this post, still think Eliezer was right about the Bank of Japan?
I do not. More accurately, I think he could be correct about the ground truth of his theory, but that he was incorrect to update as heavily as he seems to have, considering the evidence in favor is very weak.
Prime age labor force participation rate is the standard measure the econobloggers I've followed (most notably Krugman and Brad DeLong, who are part of the community also pushing for this interpretation of monetary policy) tend to use to measure economic health, and there are reasons to see it as pointing most closely to what we actually care about (that and hourly productivity, which isn't in these charts).
That's fair. FWIW, I don't follow monetary policy very closely, but I usually see people talking about unemployment, price levels, and the general labor force participation rate in these discussions, not prime age labor force participation rate. The Bank of Japan's website has a page called "Outline of monetary policy" and it states,
The page does not mention prime age labor force participation, or even employment levels.
But regardless, I think I was overconfident here. I think I'll reword the post slightly.
I think your remarks are good, but I would like to add a few more critical points as an economist. First, the RGDP is a measure that captures more of the "supply side" of an economy, so it addresses issues related to the total factor productivity of an economy. In the case of Japan, as previously noted by Prescott and Hayashi in their article on the crisis of the Japanese economy in the 1990s, there is a major problem of productivity that affects the capacity for real growth. To the surprise of people who grew up with the myth of Japanese efficiency, Japan has been the economy with the lowest labor productivity among the G7 economies for over 50 years. There is a big problem with respect to lifetime employment, difficulties with union reforms, diseconomies of scale and an extremely low number of startups. A recent research by BOJ researchers indicates that the Japanese economy has serious problems of capital accumulation and low productivity of its main companies (which have practically become cyberpunk megacorps paralyzed by the lack of incentives for innovation).
In this scenario, it is very unlikely that there will be a real positive change in the picture of real economic growth for Japan. Eliezer could advocate whatever monetary policy he wanted, but he would still probably be wrong, because the solution would be more through microeconomic reform. Perhaps an analysis from the point of view of new-classical macroeconomics is more fruitful than an analysis from the market monetarism paradigm.
Eliezer seems to be talking about trillions of dollars at stake worldwide. That's more plausible than trillions in Japan.
I disagree. Elsewhere in the chapter he says,
and later he says,
It seems like he's talking about trillions of dollars of economic growth to Japan in these passages, especially the first passage.
It's noteworthy that he also says,
This monetary expansion seems to confirm MMT that government can expand money supply without inflation when production is able to respond in kind to increased demand without inflation. THIS IS HUGE, theory previously was that taxation was only route to government spending but government can just print more Trillions, national debt essentially becomes a footnote.
Isn't it more striking that inflation didn't rise despite doubling the money supply? what would have happened if money supply didn't double???
Not an economist but it seems like deflation would have landed Japan in economic contraction rather than expansion hard to compare status quo vs resulting equilibrium from intervention likewise; actual equilibrium from intervention from resulting status quo affects.
Japan's problem is Keynes "optical illusion" problem. Banks don't lend deposits. Deposits are the result of lending/investing. It is an extraordinary delusion. Ergo, all bank-held savings are frozen. That destroys the velocity of circulation.
Japan’s “lost decade” is due to the impoundment and ensconcing of monetary savings in their banks. The BOJ has unlimited transaction deposit insurance, the Japanese save more, and keep more of their savings in their banks.
“Japanese households have 52% of their money in currency & deposits, vs 35% for people in the Eurozone and 14% for the US.”
Economists simply can’t differentiate between an individual bank and the system. The equation, the capacity of a single bank to create credit as a consequence of a given primary deposit (and newly created deposits flow to other banks), is also applicable to a nonbank, financial intermediaries. A bank: L = P (1-d) & A nonbank: L = S (1-s)
But this comparison is superficial since any expansion of credit by a commercial bank enlarges the money supply, enlarging the system, whereas any extension of credit by an intermediary simply transfers ownership of existing money within the system (a velocity relationship).
An interesting thing is that you can't print real money (gold, bitcoin, dollar if you are in Japan). Any money you can print will stop to be real ones soon, as people will exchange them into the real ones. As a result, you will have inflation in fake money but deflation in real money. Most governments who tried to print too much money has experienced it (e.g. Russia in 1990s).
Inflation is a Goodharted measure of economic growth. As soon as we start target it, it will decouple from economy.
All this has special relation to Japan which has very crazy monetary policy because of two things: tightly controlled exchange rate of yen to dollar (there are rules which prevent Japanese goods from being too cheap) and carry-trade (getting credits in yen to invest in US economy). In short, new printed yens are getting pumped in the US, not Japan economy (stock market), and too much inflation is prevented by regulation of exchange rate.
What do you mean by “real money”? What effects on the world does it have that “fake money” doesn’t? M1 in the United States increased a lot during the COVID-19 pandemic, does that mean that the US dollar is no longer “real money”?
You seem to be claiming (though correct me if I’m wrong) that expansionary monetary policy can’t achieve its objectives. What makes you believe that?
I understand that excessive money-printing that leads to very high inflation can decrease confidence in a currency and make people purchase another currency if they’re able to do so. However, that seems meaningfully different from having a central bank try to print enough money to get to ~2% YoY inflation from a baseline of zero or negative inflation.
(Note: I don't know much about monetary policy and could be confused about something.)
Dollars are real money because almost all prices in world are nominated in dollars, and it is widely used as an instrument to store value.
In Japan, excess money goes to US stock market via carry-trade, and don't contribute to inflation of money in real economy.
Can someone explain the disagree votes here?
Really interesting. I didn't know that their was a Yen-Dollar exchange rate control mechanism. That sounds like it could plausibly be a major reason for Japan's long-term economic woes. The story would go something like "late 80's, Japan doing very well. Yen set very strong vs. dollar". Then later "Japan has a recession, economy weaker. Yen prevented from falling by weird controls." Leaving Japan with an overpriced currency and thus a weakened export market and weak growth.
There also an interesting point that money experiments create parasites. In Russia in 90s there was rampant money printing and inflation, but economy declined in real term in dollars. One of the reason is that people (including one my friend) took credits in rubles in bank, convert them in dollars and just wait; after a year they exchangeв back at a better exchange rate and returned the credit, taking profit and not producing any substantial. This creates additional pressure on exchange rate and was self-perpetual mechanism.
Carry-trade in Japan is probably also a type of parasite on the near-zero interest rates.
I'm struggling to understand your first paragraph and I think this sentence has a typo?
I will try to explain with real world example. Russian government in early 90s printed money and pay a person a salary. The person took the money and exchanged them on dollars in the same day. Dollars are the real money. Many goods like electronics had prices only in dollars in stores. You can pay in dollars or can change your dollars into rubles by a new rate which grew everyday.
I read it as "People would use other forms of money for trade if the government fiat ever turns into monopoly money"