RAND and Metaculus just published Integrated Strategic Forecasting: A New Methodology to Enhance Decision Advantage, co-authored by Anca Agachi and Miriam Pasternak Jørgensen of RAND and Metaculus's Molly Hickman. The paper outlines an exploratory approach to futures and anticipatory planning that we're calling integrated strategic forecasting (ISF). You can read the whole paper here. This is a little summary.
Greater than the sum of their parts
We're in what the paper calls a permacrisis. Things are changing rather quickly and we should be using all the tools at our disposal to anticipate future events so that we can intervene more effectively. ISF builds on two familiar methodologies used in government, academia, and the private sector: scenario planning and crowdsourced forecasting.
Traditional scenario planning helps us imagine plausible, discontinuous paths the future could take and lets policymakers rehearse tradeoffs, but it doesn't tell us which futures are more likely. Crowdsourced forecasting gets more precise by assigning probabilities, “transforming amorphous uncertainty into quantifiable risk,” but on very narrow, carefully operationalized questions; not that useful in isolation.
Traditional scenario planning helps us imagine plausible, discontinuous paths the future could take and lets policymakers rehearse tradeoffs, but it doesn't tell us which futures are more likely. Crowdsourced forecasting gets more precise by assigning probabilities, "transforming amorphous uncertainty into quantifiable risk,"[1] but on very narrow, carefully operationalized questions; not that useful in isolation.
Our paper asks: How can the integration of scenario planning and crowdsourced forecasting enhance decision advantage?
The gist
Five steps, involving project managers, SMEs, forecasters and stakeholders:
Issue Identification and Scoping Discussion. Define the strategic topic, objectives, timeline, scope, and who the stakeholders are.
Scenario-Based Decomposition. Three sub-steps:
Scenario Development & Planning. Basically follow a standard scenario planning process to generate scenarios (details in Chapter 2 of the paper).
Scenario-Based Decomposition. Decompose scenarios into drivers, subdrivers and signals.[2]
Formulating Forecasting Questions. Turn signals into resolvable forecasting questions, filtered by Value of Information (VOI).[3]
Forecasting Map. Organize and prioritize questions based on their associated signals and drivers. [add a radiant figure instead of the ugly / nonsense / outdated figures in the paper?]
Reintegration and Tracking. As forecasts evolve, feed probabilities and rationales back into scenarios. When questions resolve in a way that's surprising or runs counter to some key assumptions, re-evaluate. Monitor and revisit the decomposition.
Resolution and Repetition. Brief policymakers on the updated scenarios and the way questions are trending. This step should be aligned with corporate or policy planning cycles.
An ISF forecasting map generated using Metaculus AI in Radiant, applied to the question “How will the AI landscape change now that agents have shipped as desktop products?”
Adapted from UK Government Office for Science's Futures Toolkit (2024). The most relevant drivers are more important for the policy area and more uncertain, which maps neatly onto the concept of Value of Information in forecasting.
Read the paper for a lot more detail and examples.
The work was undertaken by the RAND Forecasting Initiative, part of the RAND Global and Emerging Risks division.
RAND and Metaculus just published Integrated Strategic Forecasting: A New Methodology to Enhance Decision Advantage, co-authored by Anca Agachi and Miriam Pasternak Jørgensen of RAND and Metaculus's Molly Hickman. The paper outlines an exploratory approach to futures and anticipatory planning that we're calling integrated strategic forecasting (ISF). You can read the whole paper here. This is a little summary.
Greater than the sum of their parts
We're in what the paper calls a permacrisis. Things are changing rather quickly and we should be using all the tools at our disposal to anticipate future events so that we can intervene more effectively. ISF builds on two familiar methodologies used in government, academia, and the private sector: scenario planning and crowdsourced forecasting.
Traditional scenario planning helps us imagine plausible, discontinuous paths the future could take and lets policymakers rehearse tradeoffs, but it doesn't tell us which futures are more likely. Crowdsourced forecasting gets more precise by assigning probabilities, “transforming amorphous uncertainty into quantifiable risk,” but on very narrow, carefully operationalized questions; not that useful in isolation.
Traditional scenario planning helps us imagine plausible, discontinuous paths the future could take and lets policymakers rehearse tradeoffs, but it doesn't tell us which futures are more likely. Crowdsourced forecasting gets more precise by assigning probabilities, "transforming amorphous uncertainty into quantifiable risk,"[1] but on very narrow, carefully operationalized questions; not that useful in isolation.
Our paper asks: How can the integration of scenario planning and crowdsourced forecasting enhance decision advantage?
The gist
Five steps, involving project managers, SMEs, forecasters and stakeholders:
An ISF forecasting map generated using Metaculus AI in Radiant, applied to the question “How will the AI landscape change now that agents have shipped as desktop products?”
Adapted from UK Government Office for Science's Futures Toolkit (2024). The most relevant drivers are more important for the policy area and more uncertain, which maps neatly onto the concept of Value of Information in forecasting.
Read the paper for a lot more detail and examples.
The work was undertaken by the RAND Forecasting Initiative, part of the RAND Global and Emerging Risks division.
J. Peter Scoblic and Philip E. Tetlock, "A Better Crystal Ball: The Right Way to Think About the Future," Foreign Affairs, Nov/Dec 2020.
Richards J. Heuer Jr. and Randolph H. Pherson, Structured Analytic Techniques for Intelligence Analysis (CQ Press)
Josh Rosenberg, Ezra Karger, Avital Morris, Molly Hickman, Rose Hadshar, Zachary Jacobs, and Philip Tetlock, Roots of Disagreement on AI Risk: Exploring the Potential and Pitfalls of Adversarial Collaboration (Forecasting Research Institute, 2023)