Kahneman's Planning Anecdote


25


Eliezer_Yudkowsky

Followup toPlanning Fallacy

From "Timid Choices and Bold Forecasts: Cognitive Perspective on Risk Taking" by Nobel Laureate Daniel Kahneman and Dan Lovallo, in a discussion on "Inside and Outside Views":

In 1976 one of us (Daniel Kahneman) was involved in a project designed to develop a curriculum for the study of judgment and decision making under uncertainty for high schools in Israel.  When the team had been in operation for about a year, with some significant achievements already to its credit, the discussion at one of the team meetings turned to the question of how long the project would take.  To make the debate more useful, I asked everyone to indicate on a slip of paper their best estimate of the number of months that would be needed to bring the project to a well-defined stage of completion: a complete draft ready for submission to the Ministry of education.  The estimates, including my own, ranged from 18 to 30 months.

At this point I had the idea of turning to one of our members, a distinguished expert in curriculum development, asking him a question phrased about as follows:

"We are surely not the only team to have tried to develop a curriculum where none existed before.  Please try to recall as many such cases as you can.  Think of them as they were in a stage comparable to ours at present.  How long did it take them, from that point, to complete their projects?"

After a long silence, something much like the following answer was given, with obvious signs of discomfort:  "First, I should say that not all teams that I can think of in a comparable stage ever did complete their task.  About 40% of them eventually gave up.  Of the remaining, I cannot think of any that was completed in less than seven years, nor of any that took more than ten."

In response to a further question, he answered:  "No, I cannot think of any relevant factor that distinguishes us favorably from the teams I have been thinking about.  Indeed, my impression is that we are slightly below average in terms of our resources and potential."

Facing the facts can be intolerably demoralizing.  The participants in the meeting had professional expertise in the logic of forecasting, and none even ventured to question the relevance of the forecast implied by our expert's statistics: an even chance of failure, and a completion time of seven to ten years in case of success.  Neither of these outcomes was an acceptable basis for continuing the project, but no one was willing to draw the embarrassing conclusion that it should be scrapped.

So, the forecast was quietly dropped from active debate, along with any pretense of long-term planning, and the project went on along its predictably unforeseeable path to eventual completion some eight years later.