Scott Sumner offers some comments here FWIW, copying and pasting:

I certainly believe the BOJ policy had the effect of boosting Japan’s real GDP, but the figure cited by Yudkowsky (“trillions of dollars”) seems excessive.

A few points:

1. In the long run, money is neutral. Hence monetary stimulus won’t impact the long run level of Japan’s RGDP or employment.

2. There’s a lot of evidence that Kuroda’s policies boosted Japan’s NGDP.

So here’s the issue. How much evidence is there that faster NGDP growth boosted Japan’s real economy (and employment) for a period of time? (Alternatively, how flexible are Japanese wages and prices?)

I’d say there is substantial evidence. Japanese stocks responded as if the policy was boosting growth. Unemployment fell to levels well below the 2006 boomlet. Also, keep in mind that growth in Japan’s working age population slowed sharply in the past decade, so trend RGDP growth is slowing substantially. Growth held up better after the 2014 tax increase than after the previous (1997?) version. Thus if Yudkowsky’s evidence was too cursory, so is this critique.

To summarize, the article makes some good points, but only shows that Yudkowsky might be wrong, not that he is wrong. I still think there’s lots of evidence that he was right and the pessimists at the BOJ were wrong, even if he exaggerates the benefits.

As an aside, he mentions my name. But people with very different views on monetary policy effectiveness—such as Paul Krugman (2018)—also see the evidence as clearly suggesting that Kuroda’s policy worked to some extent.

(There’s lots more I could say, but I’m on vacation.)

Just to say it explicitly, I also don't see at all how the graphs you've shown could be interpreted to support Eliezer's interpretations. I also don't understand how Eliezer arrived at the decision to tell this story in this way based on this data.

My only background on this issue is reading Inadequate Equilibria and this post, so that's all I'm going on here.

However, I think this error undermines a significant part of Yudkowsky's thesis. This example was one of two major anecdotes that Yudkowsky presented to show that he can often know better than experts, and he cited it repeatedly throughout the book. Yet, I think he got it wrong.

It does seem plausible that the Bank of Japan thing was an error. However, I don't think that would undermine his thesis.

It's been some time since I read the book so I could be misremembering, but I remember the big idea here to basically be "consider the incentives". If there are strong incentives to be correct about something (like the price of a stock) you should be weary of disagreeing. On the other hand, if there aren't strong incentives to be correct about something, you shouldn't be too hesitant to disagree. More generally, your hesitance should be in proportion to the strength of the incentives.

I think the Bank of Japan example was meant to be illustrative of this idea, not strong evidence for it. As I understand it, the bulk of the evidence is more gears level/theoretical/deductive. Maybe something like this:

• We know that people respond to incentives.
• Therefore, the more heavily you incentivize people to be correct about something the more likely they are to actually be correct.
• Therefore, a thing where people are heavily incentivized to be correct is likely to actually be correct.
• Therefore, you should be hesitant to disagree in places where people are heavily incentivized to be correct.

Attempted paraphrase of this post:

At time1, Eliezer thought that Sumner's macroeconomic analysis was correct, and that it showed that the Bank of Japan's monetary policy was too tight, at a cost of trillions of dollars.

At time2, Eliezer wrote Inadequate Equilibria where he used this view of time1 Eliezer as one of his central examples, and claimed that events since then had provided strong evidence that it was true: Japan had since loosened its monetary policy, and their economy had improved.

Now, at time3, you are looking back at Japan's economy and saying that it didn't actually do especially well at that time, and also that [EDITED] its monetary policy never actually became loose enough to generate inflation. So Eliezer at time2, who thought that the views of time1 Eliezer had been tested and shown correct, was wrong about that.

Eliezer's views at time1 might have been correct, but we never got a clear empirical test of that, and Eliezer at time2 was wrong to claim that we had.

Further illustrating Eliezer's misplaced confidence, Sumner's view is about NGDP targeting, so the success of the BOJ's policy should be based on delivering NGDP growth, not real economic variables like RGDP growth or employment rate as Eliezer implies. They were in fact successful at this (RGDP growth + Inflation = NGDP growth; with RGDP growth continuing on trend and Inflation bucking the downtrend, that's a new NGDP trajectory, baby!). Here, with 100=March 2013 as Kuroda ascended, you can see the shift in CPI trend even before the VAT impact in April 2014. Sumner was bullish on the new BOJ policy by September 2013.

So, Eliezer, you think you have identified which econbloggers, like Scott Sumner, know better than the Bank of Japan, do you? Eliezer did identify Sumner successfully, but he got lucky. His belief in Sumner was based on a misread of Sumner's position, one that led him to wrongly believe real economic variables would supply evidence for the veracity of the theory. Further compounding the issue, while employment rate might have been readable as supportive, as Matthew Barnett points out, RGDP was not. He is overconfident and should be more humble about his approach.

Ironically, Eliezer's mistake actually more strongly makes his key point. The demand for humility Eliezer was writing about stemmed from the belief that even a very good reasoner oughtn't be able to outperform "the experts." And yet, here we have a mistaken reasoner outperforming "the experts" (at least, outperforming the hawkish experts, before they were replaced by the dovish experts who implemented the new monetary policy at the BOJ). Perhaps the case for humility is not so strong after all: "it is perfectly plausible for an econblogger to write up a good analysis of what the Bank of Japan is doing wrong, and for a sophisticated reader to reasonably agree that the analysis seems decisive, without a deep agonizing episode of Dunning-Kruger-inspired self-doubt playing any important role in the analysis." I suppose one might need to decide how interchangeable "humility" and "agonizing self-doubt" are...

Eliezer is driving an intellectual racecar when many are driving intellectual horse-and-buggies. Still needs to be vacuumed out from time to time though.

At some point in my life I was convinced by a theory of macroeconomics called modern monetary theory. I was convinced everybody was wrong about macroeconomics and I was right.

The reader might find it amusing to learn that, in stark contrast to many other issues of this type, this particular belief counterfactually cost me tens of millions of dollars.

Humbled by age, experience and debate with actual economists, no longer possessed by a Yudkowskian sense that I am right and everybody is wrong I am no longer certain I know the answer on macroeconomics. But I will point out that it is a characteristic prediction of MMT that monetary operations will have NO effect on inflation - only fiscal deficits determine inflation.

Same as DirectedEvolution, my only background is reading inadequate equilibria and this article. Does someone, after reading this post, still think Eliezer was right about the Bank of Japan?

Prime age labor force participation rate is the standard measure the econobloggers I've followed (most notably Krugman and Brad DeLong, who are part of the community also pushing for this interpretation of monetary policy) tend to use to measure economic health, and there are reasons to see it as pointing most closely to what we actually care about (that and hourly productivity, which isn't in these charts). 

bad monetary policy that cost Japan trillions of dollars in real economic growth.

Eliezer seems to be talking about trillions of dollars at stake worldwide. That's more plausible than trillions in Japan.

An interesting thing is that you can't print real money (gold, bitcoin, dollar if you are in Japan). Any money you can print will stop to be real ones soon, as people will exchange them into the real ones.  As a result, you will have inflation in fake money but deflation in real money. Most governments who tried to print too much money has experienced it (e.g. Russia in 1990s). 

Inflation is a Goodharted measure of economic growth. As soon as we start target it, it will decouple from economy.

All this has special relation to Japan which has very crazy monetary policy because of two things: tightly controlled exchange rate of yen to dollar (there are rules which prevent Japanese goods from being too cheap) and carry-trade (getting credits in yen to invest in US economy). In short, new printed yens are getting pumped in the US, not Japan economy (stock market), and too much inflation is prevented by regulation of exchange rate.