That's a really great way to think about it- thank you for that metaphor.

I'm sorry that I missed this. 

I don't have a great answer for you. It is largely speculation about counterfactual benefits. I would like to delude myself into thinking it's good counterfactual speculation, but that's probably just my ego talking. 

I'm mostly basing this around things that continue to be problems for years. For instance, I work as an AI research engineer at a large industry lab, and a common task is to launch experiments that use O(100) GPUs. We have a system that automatically schedules these jobs on different data centers depending on availability. It does not have the ability to choose a data center that satisfies constraints on both spot instances and on-demand instances. The scheduler can only do one of these. As a result, we have to manually choose cells. It has been like this for >4 years. 

I have a bunch of additional examples like this that have gone unsolved for years. This is a log way of saying that, to directly answer your question, I don't have much more than speculation. 

Progress studies is really interesting to me, because if one actually believes that it has a >0 chance of working, the EV is so huge that we should be directing huge amounts of funding towards it. As we are not [0], this implies that most organizations/people either think it has no chance of working, or haven't heard of it. 

There's an analogy to be made to internal tooling at most companies. All companies I've been part of radically underfund their internal tooling, even though it has a direct and quantifiable ROI. I don't know why this is. Increasing the 2nd derivative is so massively valuable that it seems obviously worth doing.

[0]: Where we = the government, private donors, universities.