Note: This post is part of my series of posts on forecasting and related topics, related to my contract work for MIRI. While I think the post would interest a sufficiently large fraction of the LessWrong readership for it to be worth posting, it's not of as much general interest as some of my other recent posts. If business strategy and planning don't interest you as topics, this post may not be for you.
I've been reviewing forecasting and related domains as part of contract work for MIRI. One of the related domains I'm looking at is scenario planning (Wikipedia), also known as scenario thinking or scenario analysis. To understand scenario planning, I read the book The Art of the Long View by Peter Schwartz (Amazon) (a fairly interesting read, though whether you consider it value-for-money depends on whether strategy planning and thinking about the future as a whole interest you) and his follow-up book Learnings from the Long View (Amazon). I also skimmed Scenario Planning in Organizations by Thomas J. Chermack (Amazon), which is somewhat drier but is more useful for people who have some understanding of scenario planning and want to implement it in full.
In this post, I discuss some aspects of scenario planning, but the post is not intended to be a standalone summary of scenario planning. For that purpose, I recommend reading the overviews of scenario planning listed below. In this post, I'll look at examples of scenario planning in diverse domains, and I'll consider the relationship between scenario planning and the more conventional, quantitative approach to forecasting.
Overviews of scenario planning
A quick summary of scenario planning
Scenario planning involves the generation of (usually two or three) scenarios for how the future might transpire. Here are some typical aspects of scenario planning:
Who uses scenario planning?
Scenario planning caught on after the 1973 oil shock; prior to that, only Shell and GE engaged in it (see the Bradfield et al. overview for more). A 1981 survey by Linneman and Klein found three predictors of whether a company used scenario planning:
Big successes of scenario planning
Some examples of scenario planning
I looked on the Internet for scenario planning writeups across diverse domains that were publicly available. I list some examples below. The focus is on what I was able to find on the Internet by using a range of search phrases, and the list below is unlikely to be representative of scenario planning.
Does scenario planning involve forecasting?
There are many ways in which scenario planning overlaps with forecasting:
Evaluation of the utility of scenario planning (some random thoughts)
I wrote above: "The utility [of scenario planning] is in how it prepares the people undertaking the exercise for the relevant futures. One way it could so prepare them is if the early indicators of the scenarios are correctly chosen and, upon observing them, people are able to identify what scenario they're in and take the appropriate measures quickly."
What does research say about scenario planning? My impression is that there is very limited research, and it tends to be positive about the effect of scenario planning on long-range development, but there is considerable uncertainty. Bradfield et al. quote Schnaars as saying that there is "a small body of research based on empirical studies of related topics, which ‘offer some evidence as to the value of scenarios’ as a long range planning tool."
In general, evaluating scenario planning seems difficult, because unlike the case of forecasting, where the accuracy of the forecast is a good first proxy for utility, scenario planning does not lend itself to that sort of evaluation. The examples that Schwartz lists provide some anecdotal evidence. But there is the obvious selection versus treatment issue: maybe the pioneers of scenario planning were selected as the sort of people who could plan better for the future, and the scenario planning exercise itself wasn't useful. The selection versus treatment issue could be resolved by looking at whether business planning has become more efficient on the whole as scenario planning has come to be more widely used (because the continued spread of scenario planning is probably a treatment rather than a selection effect). But isolating scenario planning as a causal factor, when so many other aspects of business strategy are changing, is hard.
Another possible approach: how different would the world be today if people didn't use scenario planning? The IPCC reports on climate change may just consider one "official forecast" with a margin of uncertainty. Energy companies might still have an Official Future (again with an expressed margin of uncertainty, but without a discrete set of alternative futures). Based on my (non-expert) assessment, it seems that the quality of business strategy and policy insight in such a world would be worse than in the current world. But perhaps if scenario planning hadn't been developed, other ways of thinking about the future would have caught on more. The upshot is that I tentatively think scenario planning has been useful, but I don't see a clear way of demonstrating this in a scientifically rigorous manner, given the nature of the beast.