LESSWRONG
LW

Model-Based Policy Analysis under Deep Uncertainty — LessWrong

This post is based on my introduction talk at EAGxBerlin 2022. It is intended for policy researchers who want to extend their tool kit with computational tools. I show how we can support decision-making with simulation models of socio-technical systems while embracing uncertainties in a systematic manner. The technical field of decision-making under deep uncertainty offers a wide range of methods to account for various parametric and structural uncertainties while identifying robust policies where we want to optimize for multiple objectives simultaneously for particularly vulnerable scenarios.

Summary

Real-world political decision-making problems are complex, with disputed knowledge, differing problem perceptions, opposing stakeholders, and interactions between framing the problem and problem-solving.
Modeling can help policy-makers to navigate these complexities.
Traditional modeling is ill-suited for this purpose.
Systems modeling is a better fit (e.g., agent-based models).
Deep uncertainty is more common than one might think.
Deep uncertainty makes expected-utility reasoning virtually useless.
Decision-Making under Deep Uncertainty is a framework that can build upon systems modeling and overcome deep uncertainties.
Explorative modeling > predictive modeling.
Value diversity (aka multiple objectives) > single objectives.
Focus on finding vulnerable scenarios and robust policy solutions.
Good fit with the mitigation of GCRs, X-risks, and S-risks.

Complexity

Complexity science is an interdisciplinary field that seeks to understand complex systems and the emergent behaviors that arise from the interactions of their components. Complexity is often an obstacle to decision-making. So, we need to address it.

Ant Colonies

Ant colonies are a great example of how complex systems can emerge from simple individual behaviors. Ants follow very simplistic rules, such as depositing food, following pheromone trails, and communicating with each other through chemical signals. However, the collective behavior of the colony is highly sophisticated, with complex networks of pheromone trails guiding the movement of the entire colony toward food sources and the construction of intricate structures such as nests and tunnels. The behavior of the colony is also highly adaptive, with the ability to respond to changes in the environment, such as changes in the availability of food or the presence of predators.

Examples of Economy and Technology

Similarly, the world is also a highly complex system, with a vast array of interrelated factors and processes that interact with each other in intricate ways. These factors include the economy, technology, politics, culture, and the environment, among others. Each of these factors is highly complex in its own right, with multiple variables and feedback loops that contribute to the overall complexity of the system. For example, the economy is a highly complex system that involves the interactions between individuals, businesses, governments, and other entities. The behavior of each individual actor is highly variable and can be influenced by a range of factors, such as personal motivations, cultural norms, and environmental factors. These individual behaviors can then interact with each other in complex ways, leading to emergent phenomena such as market trends, economic growth, and financial crises. Similarly, technology is a highly complex system that involves interactions between multiple components, such as hardware, software, data, and networks. Each of these components is highly complex in its own right, with multiple feedback loops and interactions that contribute to the overall complexity of the system. The behavior of the system as a whole can then be highly unpredictable, with emergent phenomena such as network effects, disruptive innovations, and the potential for unintended consequences.

Socio-Technical Systems

In the context of EA, it might be particularly interesting to look at socio-technical systems. These are complex systems that involve the interactions between social and technical components, such as people, organizations, and technology. One key characteristic of these systems is non-linearity, which means that small changes or disruptions in one part of the system can have significant and unpredictable effects on other parts of the system. This non-linearity can contribute to the overall complexity of the system, making it difficult to understand and control. The non-linear and complex nature of socio-technical systems presents a significant challenge for policymakers. Policies and regulations designed to address specific issues or problems may have unintended consequences or fail to achieve their desired outcomes due to the complex interactions within the system. Additionally, policymakers may not have access to complete or accurate information about the system, making it difficult to make informed decisions. The multiple stakeholders involved in socio-technical systems may also have conflicting interests and goals, making it challenging to reach a consensus and implement effective policies. Furthermore, the rapid pace of technological change can exacerbate the complexity of these systems, as new technologies can disrupt established social and economic structures, creating new challenges for governance. To effectively address the complex challenges posed by socio-technical systems, policymakers need to adopt new approaches that embrace complexity and uncertainty, such as the methodology that I'm advocating in this post.

Modeling

We care about the state of the world. Policy-making can affect this state. However, navigating through a space of endless possibilities renders it hard to find good policy solutions to lead us to desirable world states. Intuition, or relying on personal experience and judgment, can be a useful tool in many situations, but it can also be unreliable and insufficient when it comes to understanding and acting on complex systems like the one described above. Ideally, we would like to see our policy-makers use evidence-based methods to support their decision-making process. Using formal modeling can be a way to offer such support.

What is Modeling?

But what is modeling actually? Modeling is the process of creating a simplified representation of a natural system that captures the key components, interactions, and behaviors of the system. This representation is encoded into a formal system, such as a computer program or mathematical equation, that can be used to simulate the behavior of the system under different conditions. Through the process of simulation, the formal system can be decoded back into a natural system, allowing policymakers and researchers to gain insights into the behavior and potential outcomes of the real-world system.

Why is Modeling Useful?

Modeling can be a powerful tool for understanding and managing complex systems, as it allows us to formalize our understanding of a system and explore different scenarios and outcomes. Formalization forces us to define and articulate our mental models of a system, which can expose any contradictory assumptions or gaps in our understanding. This process can help us identify areas where we need more data or research, or where our assumptions need to be revised. By formalizing our mental models and combining them with observed data, we can develop more accurate and comprehensive models of complex systems, which can be used to inform policy decisions and interventions.

Despite the gaps in our knowledge and the limitations of modeling, there is value in attempting to model complex systems. We cannot wait until we have perfect information to make policy decisions, as urgency often dictates that we must act even in the face of uncertainty. Modeling allows us to explore different scenarios and evaluate the potential outcomes of different policy options, even when we are uncertain about the behavior of the system.

As George Box famously said,

"All models are wrong, but some are useful."

While no model can perfectly capture the complexity and nuances of a real-world system, some models can still be useful in providing insights into the behavior of the system and potential outcomes of policy decisions. By embracing the imperfect nature of modeling and using it as a tool to refine and improve our understanding of complex systems, we can make more informed and effective policy decisions.

Traditional Modeling

I would like to distinguish two paradigms of modeling. Something that I call traditional modeling on the one hand, and systems modeling on the other hand.

Traditional modeling, particularly in economics, is typically based on the assumption of rational actors making rational choices based on a cost-benefit analysis. This approach, known as rational choice theory, assumes that individuals will act in their own self-interest and maximize their utility based on a set of preferences and constraints. The resulting models are often based on elegant equations and mathematical frameworks and rely heavily on aggregation and simplification to make complex systems more manageable. This approach assumes a top-down view of the world, with individual behavior aggregated into macro-level patterns and trends.

However, this particular modeling style is ill-suited for dealing with complexity and uncertainty. The assumptions of rationality and self-interest are often oversimplified and do not accurately capture the full range of motivations and decision-making processes of individuals and groups. Additionally, the reliance on aggregation and simplification can obscure important details and interactions within the system, leading to inaccurate or incomplete models. This is particularly problematic when dealing with complex systems, where the interactions between different components and actors can be highly non-linear and unpredictable.

Furthermore, traditional modeling approaches often ignore the role of uncertainty and the potential for unexpected events or disruptions to occur. This is particularly problematic in complex systems, where small changes or perturbations can have significant and unexpected effects on the behavior of the system. The top-down view of traditional modeling approaches may also fail to capture the emergent behaviors and feedback loops that can arise from the interactions of multiple actors and components within the system.

Systems Modeling

So, while traditional modeling approaches based on rational choice theory and top-down aggregation have their strengths, they are ill-suited for dealing with complexity and uncertainty. To effectively model complex systems, new approaches are needed that embrace complexity and uncertainty and incorporate more nuanced and realistic assumptions about individual behavior and decision-making. The umbrella term of systems modeling offers a plethora of alternative modeling sub-paradigms. This includes agent-based modeling, discrete event simulation, and system dynamics modeling^[1]. I consider agent-based modeling to be an especially potent paradigm that I would like to elaborate on.

Agent-Based Modeling

Agent-based modeling is an approach to modeling complex systems that focuses on the interactions and behaviors of individual agents within the system. In an agent-based model, each agent is represented as an individual entity with its own set of characteristics, behaviors, and decision-making processes. These agents interact with each other and with their environment, leading to emergent patterns and behaviors at the macro level.

The underlying assumption of agent-based modeling is that complex systems can be understood and predicted by modeling the interactions and behaviors of individual agents. This approach allows for more realistic and detailed modeling of individual behavior and interactions within the system and can capture the non-linear and unpredictable behavior that often characterizes complex systems.

Agent-based modeling typically involves the following steps: defining the agents, specifying their behaviors and decision-making processes, defining the environment, and simulating the behavior of the system over time. The behavior of each agent is based on a set of rules or algorithms that govern their decision-making processes, and the interactions between agents are modeled using communication channels or networks.

Example: Epidemic Disease Modeling

A relevant example is the epidemic disease model as for example shown here^[2]. Agent-based modeling allows for the modeling of individual behavior and interactions within a population. In an agent-based model of an epidemic, each agent represents an individual within the population and is assigned a set of characteristics, such as age, health status, and social network. The behavior and decision-making processes of each agent are then modeled based on a set of rules or algorithms that capture their interactions with other agents and with their environment. Using an agent-based model, researchers can simulate the spread of an epidemic through a population, and evaluate the potential effectiveness of different intervention strategies, such as vaccination, quarantine, or social distancing measures. The model can also be used to explore the impact of different variables on the spread of the disease, such as the rate of transmission, the incubation period, or the effectiveness of interventions. Agent-based modeling can provide several advantages over traditional epidemiological models, which often rely on simplified assumptions about population behavior and interaction. By modeling the behavior of individual agents, agent-based models can capture the heterogeneity and complexity of real-world populations, and account for the effects of social networks, geographic location, and other factors that can influence the spread of the disease.

Deep Uncertainty

There have been recent posts and other EA engagement with complexity science and systems modeling^[3]. So, why add another? Using systems modeling is only one step of the research methods that I'm recommending EA researchers to use. The other step consists of conducting decision-making under deep uncertainty (DMDU). First, we need to address the question of what deep uncertainties are. We talk about deep uncertainties when parties to a decision do not know, or cannot agree on^[4]

the system model that relates action to consequences,
the probability distributions to place over the inputs to these models, or
which consequences to consider and their relative importance.

The first point (system model mapping) relates often to structural uncertainty in which we do not know how a natural system maps input and outputs. We are not certain about (some of) the underlying structures or functions within the formal model. The third point (consequence and relative importance) refers to the standard approach of oversimplification and aggregation and putting some a priori weights on several objectives. We will elaborate on this point further down the text. The second point (probability distributions) relates to that we do not know how likely particular values for particular input variables are. This is especially the case when we are looking at very rare events for which a frequentist probability is usually tricky. To elaborate on this, consider the table below. In this context, the literature distinguishes 5 levels of uncertainty^[5].

Uncertainty Levels: Deep uncertainty is represented by the blue-shaded columns.

On the first level, you are looking at virtual certainty. It's (basically) a deterministic system. Level 2 uncertainty is the kind of uncertainty that most people refer to when they say uncertainty. It's the assumption that you know the probability distribution over alternate futures and is on a ratio scale^[6]. Level 3 uncertainty is already rarer in its usage. It refers to alternate futures that you can only rank by their probabilities and is on an ordinal scale. In this case, we do not know the exact probabilities of future events but only which ones are more likely than others without indicating how much more likely.

Deep uncertainties refer to levels 4 and 5 uncertainties. Level 4 uncertainties refer to a situation in which you know about what outcomes are possible but you do not know anything about their probability distributions, not even the ranking. It's just a set of plausible futures. This is an extraordinarily important consideration! If true, this breaks expected-utility thinking.

Expected-Utility Thinking is in Trouble

In the absence of probability distributions, expected-utility thinking becomes useless, as it relies on accurate assessments of probabilities to determine the expected value of different options. How do you calculate expected utility if you do not know what to expect? Spoiler alert: You don't! Some would make the argument that we often do not know the probability distributions over many external factors that affect our systems. Sure, we know how likely a dice role is to yield a 5. But do we really know how likely it is that AGI will be devised, tested, and publicly announced by 2039? As Bayesians, we might tend to think that we can. But how useful are such kinds of probability distributions? A more modest position would consist in acknowledging that we do not know these probability distributions. We might argue that we know plausible date ranges. But this doesn't sit well with expected-utility thinking. Hence, deep uncertainties pose quite a challenge which we will address a bit later.

(Level 5 uncertainty refers to unknown unknowns that the suggested framework of DMDU can partially address with adaptive methods^[7].)

Real-World Political Decision-Making

Real-world political decision-making is a complex and multi-faceted process that involves multiple actors at different scales and levels of governance. This system is characterized by deep uncertainties and multiple competing interests and values, making it difficult to arrive at effective and sustainable policy decisions.

One key challenge of political decision-making is the inherent uncertainty of our knowledge. In many cases, we may not have complete or accurate information about a problem or its potential solutions, making it difficult to arrive at a consensus or make informed decisions. Additionally, different stakeholders may have different interpretations of the available information, leading to disagreements and conflicts over how to address a problem. Another challenge is the presence of multiple stakeholders with partially opposing values and interests. These actors may have different perspectives on a problem or different priorities for its solution, leading to disagreements and conflicts over policy options. Additionally, different actors may have different levels of power or influence within the decision-making system, which can affect the outcomes of the decision-making process. Yet another challenge is the existence of multiple ways of perceiving a problem or defining its scope. The framing or problem formulation can significantly affect the outcome of the decision-making process, as it can shape the way that stakeholders perceive and approach the problem. This can lead to different policy options or solutions being favored based on how the problem is framed.

Given all these challenges, what methods are we usually using to address real-world political decision-making? The short answer: We use inappropriate methods. Let's have a look at the diagram below.

We have a space spanned by two dimensions: knowledge and values. We are looking at different levels of knowledge. The further we move to the right, the more uncertain things are getting. The vertical axis indicates what type of values we care about. The lowest one – a single measure of goodness – represents a single value, e.g., discounted aggregate welfare based on consumption as is typical in economics. The next level is a so-called multi-attribute utility which translates a set of relevant variables into a single utility value. Let's say, you care about the values of welfare, equality, safety, and freedom. And let's assume you can somehow operationalize them in a model. In the framework of multi-attribute utility, you create a function that takes these as inputs and it outputs a utility value (or ranking over a bunch of futures). This is also a common way of aggregation and is considered by many as a good way how to handle multiple values. The last axis label is value diversity which does not aggregate your variables. You simply look at and optimize for various, potentially orthogonal values at the same time.

The standard decision analysis techniques assume that real-world political decision-making falls into the red area. We are certain about how systems work or we at least know the probability distributions over the inputs. And at the same time, a single measure or multi-attribute utility are appropriate ways to address the challenges of the real world. However, given our elaborations from above, real-world problems are more complex, exhibit deep uncertainties, and have many stakeholders with partially opposing values that are not easily aggregated. The obvious next question is: What methods can we use to address such problems? The answer is Decision-Making under Deep Uncertainty (DMDU).

Decision-Making under Deep Uncertainty

The DMDU process involves repeatedly considering different potential strategies, understanding their weaknesses, and evaluating the trade-offs between them to ultimately make informed decisions. It is commonly used by researchers to assess different options, but it is also utilized as a tool for decision support, focusing on assisting decision-makers in identifying and creating new options that are more robust. These options often involve adaptive strategies that can change over time in response to new information. DMDU can also be used to manage group decision-making in situations where parties have conflicting assumptions and values.

XLRM Framework

In order to explain better how DMDU works, let us look at a strongly abstracted standard model. Model information can be structured within the XLRM framework which was developed by RAND researchers^[8]:

X: Exogenous Uncertainties. These are variables that a decision-maker has no control over but will, however, still affect the system and therefore the performance of policies. All uncertainty variables span the uncertainty space. A point in that space is called a scenario.
L: Policy Levers. These are variables that a decision-maker has control over and with which they can affect the system. All lever variables span the policy space. A point in that space is called a policy.
R: Relations: This component refers to the inner workings of the system, describing a mapping between X and L on the one hand, and the metrics (M) on the other hand.
M: Performance Metrics: These are variables that a system outputs and that a decision-maker cares about.

The diagram below shows how levers and uncertainties are inputs to the model which is defined by its relations. This is model-paradigm-agnostic representation. The relations can be expressed by traditional modeling ways via simple mathematical equations, or by the rules that govern a particular agent-based system. The outputs of the model are our performance metrics.

Let's talk definitions. Within decision-making, a policy is a point in a lever space. If you have 2 orthogonal actions lever you can take, then a value combination of these two levers makes a policy. Let's say, for example, you can decide the savings rate for your household. It could range from [0.0, 1.0]. At the same time, you might want to decide how much time you want to spend on reading per week with a range of [0, 10] hours. These two actions span your policy space. Any combination is a possible policy.

Exogenous uncertainties work in a similar way. Every variable in your system that is uncertain spans together an uncertainty space. For example, you don't know how much money you will have available (maybe you are gambling or day trading). But you think that the range is between €[1000,4000]. And you don't know for how long your friend will stay over on the weekend, maybe something in the range of [Thursday, Monday]. Any given combination determines a scenario.

Your metrics can be various independent objectives that you have, e.g., number of pages read, quality of spent time with your friend, satisfaction with your savings, etc.

The relations would determine how to map the inputs with the outputs.

Predictive Modeling

As I see it, there are two different policy frameworks in this context.

In a predict-and-act policy framework, predictive models are used to anticipate the potential outcomes of different policy options or interventions, allowing policymakers to make more informed and effective decisions. Predictive modeling typically involves the following steps: defining the system and its components, specifying the relationships and interactions between the components, and simulating the behavior of the system over time. The resulting model can be used to explore different scenarios and evaluate the potential outcomes of different policy options or interventions. What is usually done by the modeler is a tweaking of single parameters and testing a handful of scenarios. This kind of work is quite manual, cumbersome, and often of limited use. Especially, as they are often caught by surprise.

What they often do is simply ignore uncertainties. Instead of using a full range (see the above example), they just collapse it into a single number. This is of course very problematic. When dealing with the real world, that makes little sense as some assumptions might likely be wrong. And in this case, your resulting policy recommendations become obsolete. Another way how predictive modelers handle deep uncertainties is summarized in the next meme.

Some predictive modelers vary their uncertain variables slightly, run their simulations a few times, and average the results. I guess, there is no need to emphasize how problematic this approach can be.

Generally, modelers do not have a magic crystal ball to look into the future. Ignoring uncertainties is not a sustainable solution. If you still want to use systems modeling, you might want to look for an alternative to predictive modeling.

Explorative Modeling

So, what's the alternative? Exploratory modeling is the use of computational experimentation to explore the implications of varying assumptions about uncertain, contested, or unknown model parameters and mechanisms. Here, we are talking about an explore-and-adapt policy framework. What we can do a bit more concretely is the following: Take your model and embed it in an optimization setup. Instead of manually changing some input parameters (policy levers and uncertainties) to see how they affect your metrics, you tell your optimizer to optimize your metrics and find the corresponding ideal policy levers. It is important to note that you do not want to maximize some kind of expected future performance. You want to minimize plausible future regret.

This approach makes it easy to avoid hand-picking values for your variables as you can explore the entire space. The results will be more useful and robust. As reasonable prediction is virtually impossible, exploration is preferred over prediction.

Multiple Objectives

Why would we need multiple objectives? We have touched upon this issue in a previous section. However, let us give it a bit more attention.

Arrow's Paradox, also known as Arrow's Impossibility Theorem, is a concept in social choice theory proposed by economist Kenneth Arrow in 1950. The theorem states that it is impossible to create a voting system that satisfies a set of fairness criteria when there are three or more options to choose from. These fairness criteria include unrestricted domain, non-dictatorship, Pareto efficiency, and independence of irrelevant alternatives. In simple terms, Arrow's Paradox suggests that no perfect voting system exists that can convert individual preferences into a collective, ranked preference list without violating at least one of these criteria. Disaggregation of multiple objectives in decision-making can help address Arrow's Paradox by breaking down complex decisions into smaller, more manageable components. By doing so, decision-makers can focus on one criterion at a time, allowing for more transparent and fair evaluations. This process enables the comparison of individual objectives using specific metrics, reducing the chance of preference inconsistencies that may arise in aggregated decisions.

Combining incommensurate variables in decision-making is akin to comparing apples and oranges, as these variables represent different dimensions and cannot be directly compared. Incommensurate variables are quantities that cannot be expressed in the same unit or do not share a common scale. When attempting to combine such variables, decision-makers may face difficulties in balancing the trade-offs between them, leading to potentially biased or suboptimal outcomes.

Take, for example, economic consumption and climate damages in integrated assessment models. Although they share the same unit (USD$ usually), it can be troublesome to conflate them into a single objective. Climate change is a complex, nonlinear process that involves multiple feedback loops and interactions. There is significant uncertainty associated with the magnitude and timing of future climate impacts, which makes it difficult to accurately quantify the damages caused by climate change. Beyond that, the impacts of climate change are not evenly distributed across the population or the world. Some regions and populations are more vulnerable to the effects of climate change than others. Therefore, aggregating economic consumption and climate damages into a single variable may obscure important distributional impacts and equity considerations. Often, it might be better to consider separate objectives to avoid such pitfalls.

Generally, premature aggregation can be bad in general because it can oversimplify complex phenomena and obscure important details and nuances. Premature aggregation can result in the loss of important information about the underlying factors that contribute to the data. This loss of information can lead to biased or incomplete assessments of the data and mask important trends or patterns. Aggregation can oversimplify complex phenomena by collapsing multiple dimensions into a single measure or index. This oversimplification can result in inaccurate or misleading conclusions and mask important trade-offs and uncertainties. And eventually, aggregation can also hide underlying assumptions and modeling choices that may influence the results of the analysis. These hidden assumptions can make it difficult to compare the results of different models or assess the robustness of the results.

At this point, you are probably asking yourself, how do we optimize for multiple objectives at the same time? The answer is: We use multi-objective evolutionary algorithms. That is at least the standard practice in the field of DMDU.

Usually, you have only one objective which looks something like this:

You have three outcomes for one objective. It's clear which outcome is dominating. But if we add another objective, things become more complicated – but still doable!

We can see that both, C and D, dominate A and B. But neither C nor D dominates each other. We consider outcomes C and D to be part of the so-called Pareto-front.

We can use these concepts in evolutionary algorithms to determine better and better policy solutions while optimizing multiple objectives simultaneously. Usually, we end up with a set of Pareto-optimal solutions that exhibit different trade-offs over the objectives.

Example Method: MORDM

Let's make it even more concrete by looking at a particular method. Within the DMDU framework, there is an approach that is called multi-objective robust decision-making (MORDM)^[9]. An abstraction of the model is shown below.

First, we specify the parameters and ranges of your model. Then, we need to determine the scenarios that we want to run the optimizations on later. As we admit that we do not know the probability distributions over our uncertainties, we can sample particular scenarios from the uncertainty space. DMDU is intended to consider worst-case scenarios and find robust policy solutions. That is why we need to identify particularly vulnerable scenarios. That means that we are looking for a combination of uncertainty values that impact our metrics in a very bad way. For this purpose, we usually use some kind of AI algorithm. If we find a handful of such scenarios that are sufficiently vulnerable and diverse, we can to the next step of the MORDM process – policy discovery. Given a vulnerable scenario and a set of objectives with their desired optimization direction, we use multi-objective evolutionary algorithms to find a set of Pareto-optimal policies. (We can do this in parallel for each vulnerable scenario.) In the next step, we sample a wide range of scenarios from the uncertainty space and re-evaluate how well our Pareto-optimal policies perform. They were optimal for the chosen reference scenarios. But that does not entail that they will be optimal or even okay in other random scenarios. However, usually, we can identify a few policies that still perform well enough in many different scenarios. If not, we feed back this information to zoom in on particular subspaces of the uncertainty space and re-iterate the process to find better and better solutions for vulnerable scenarios that perform also well in more favorable scenarios. In the end, we conduct a trade-off analysis.

Scenario Selection

Instead of fixing all uncertainties, or handcrafting a handful of scenarios, what we usually do is sample thousands of scenarios. Consider for example an uncertainty space of only two uncertainty variables. Sampling from that space with uniform distributions per axis (or slightly more sophisticated sampling methods such as Latin hypercube sampling) will result in the figure below.

Every dot represents one scenario. What we usually want to do, is find the vulnerable subspace, i.e. the space where we find the scenarios which lead to the most bad outcomes (e.g., defined by some constraints on our objectives). The red dots in the figure below represent such bad scenarios. The red box marks the vulnerable subspace. Note that there are also okay scenarios in that box but that is fine. We do not need to define a subspace that contains only scenarios with bad outcomes. A rough subspace where more scenarios of bad outcomes are predominant is sufficient.

What we do next is select particular scenarios in that vulnerable subspace that represent the subspace in the best way. So, we are looking for scenarios that are vulnerable but also maximally diverse. In the figure below, you see these scenarios as black dots. (These have been randomly put in place for this example. There are formal methods to find these scenarios but we will not discuss them here.)

These three scenarios would represent our reference scenarios. We would use these for our policy discovery process with our evolutionary algorithms to find Pareto-optimal policy solutions. From there, we continue with reevaluation under deep uncertainty, scenario discovery, etc.

This should just represent a teaser for what it is like to use this kind of methodology.

Example Plots

The visual outputs can be quite diverse. But I would like to show 3 examples.

The first example is a so-called parallel-axes plot^[10]. Each vertical axis represents an objective. Values at the top of the axis are considered more desirable. A line that runs through all axes is one particular outcome of a model run. For example, the black line A exhibits undesirably low utility, desirably low warming above 2 degrees Celsius, medium damage costs, and a not-so-desirable amount of mitigation costs. This shows a trade-off. Line B exhibits a different trade-off. If you look closely, you can see many more lines. And from this, you can conclude more general patterns and trade-offs. (All these outcomes are based on optimal policies in various scenarios.)

There are also other plots such as subway plots which depict tipping points in the future to indicate a change to a different policy^[11]. Each colored line represents one policy. Every ring is a transfer station to a different policy. As we can observe, it is not always possible to transfer to any other policy. And sometimes there are terminal points that indicate that we cannot transfer at all anymore.

These plots are supposed to be teasers for how visual deliverables could look when using DMDU techniques.

Potential Drawbacks

The three main drawbacks of using DMDU are depicted in the memes below.

Con: Encoding a Complex World into Simple Models

The first drawback is one that refers more to encoding complex models in general. Complex models often involve a large number of variables and parameters, making it difficult to represent the full complexity of the system in a formal model. This can lead to oversimplification or abstraction of important details, which may affect the accuracy or validity of the model. Even when we use DMDU and embrace uncertainties instead of ignoring them, we still have to make a lot of assumptions and decisions throughout the modeling process. Furthermore, complex models involve non-linear or dynamic interactions between different variables, which can be difficult to capture in a formal model. This can lead to the development of models that are overly simplified or do not accurately reflect the true behavior of the system. Verification and validation can be very hard. Moreover, encoding complex models can be time-consuming and resource-intensive, requiring significant expertise and computational resources to develop and test the model. This can limit the applicability of the model or make it difficult to develop models that are accessible to a wide range of stakeholders.

Con: Stakeholder Cognitive Overload

Predictive modelers can offer point predictions to stakeholders. That is easy to understand and might feel actionable. The DMDU outputs are of explorative nature and involve a lot of scenarios, caveats, trade-offs, and insights. Communicating this can be a challenge. If not done with care, all the analysis might be futile, if the stakeholder is unable or unwilling to be walked through the complexities of the model and the trade-offs of available policy options.

Con: Compute

Running for example an agent-based model can require a lot of computing. Some models run in a few minutes, some might need a few hours on a laptop. Running optimizations with such models with many many iterations can take weeks, months, or years. This depends of course on the complexity of your model, how many agents you have, how many interactions they have, how many objectives, uncertainties, and policy levers you use, and what kind of datatypes you use (are you uncertainties binary, nominal, a finite set of values, or range between two floats?). Given enough resources, we should be able to overcome this particular problem.

Conclusion

I hope that this post was able to show you why I think this approach could be so valuable. The world is complex. Policy-making is hard. Modeling can help if we use systems modeling in particular. Decision-making under deep uncertainty is an excellent tool to leverage these models even better. I think that the following meme is the most important one in this post and it is what I like the reader to take away from.

Decision-making under deep uncertainty seems to me like a natural fit with EA's efforts to address global catastrophic risks, existential risks, and suffering risks. We are looking purposefully at very undesirable future scenarios and attempting to find policy solutions that would be robust, rendering worst-case scenarios less bad. In other words, we can use these techniques to strengthen societal resilience. I see the most relevant application areas of this methodology in:

Biorisk
Economics
Improving Institutional Decision-Making
Climate Change
Nuclear War
AI Governance
Longtermism

I would be happy to talk to anyone that has remaining questions or is interested in using this methodology for EA-motivated projects. I'm particularly interested in applying model-based policy analysis under deep uncertainty on AI governance.

Additional Resources

On Modeling

On Deep Uncertainty

Some papers using the methodology of decision-making under deep uncertainty (DMDU):

Climate Change Mitigation and Adaptation: DMDU is commonly used in climate change policy-making, where the long-term consequences of policy decisions are uncertain and complex.
Healthcare Resource Allocation: In healthcare, DMDU is used to make decisions on resource allocation, such as the allocation of funding for disease research and the development of new treatments.
Disaster Response Planning: DMDU is also used in disaster response planning, where decision-makers must anticipate and prepare for multiple potential outcomes in the face of extreme uncertainty.
Environmental Protection: Environmental policy-making often involves DMDU, such as decisions on the conservation and management of endangered species, and management of ecosystems and biodiversity.

^{^}
An excellent resource for exploring various model paradigms is Scott E. Page's book The Model Thinker: What You Need to Know to Make Data Work for You (2018).
^{^}
You can use this link to play around with the parameters and observe the emerging macro behaviors.
^{^}
See for example: What complexity science and simulation have to offer effective altruism, Simulation models could help prepare for the next pandemic, Complexity Science, Economics and the Art of Zen.
^{^}
Walker, W. E., Lempert, R. J., & Kwakkel, J. H. (2012). Deep uncertainty. Delft University of Technology, 1(2).
^{^}
Walker, W. E., Lempert, R. J., & Kwakkel, J. H. (2012). Deep uncertainty. Delft University of Technology, 1(2).
^{^}
E.g., a fair dice has a uniform distribution of 1/6 over all 6 outcomes. Mind that there does not have to be a uniform distribution to be considered level 2 uncertainty.
^{^}
However, I will not focus on adaptive methods in this post. For more information, see for example:
Maier, H. R., Guillaume, J. H., van Delden, H., Riddell, G. A., Haasnoot, M., & Kwakkel, J. H. (2016). An uncertain future, deep uncertainty, scenarios, robustness and adaptation: How do they fit together?. Environmental modelling & software, 81, 154-164.
Walker, W. E., Haasnoot, M., & Kwakkel, J. H. (2013). Adapt or perish: A review of planning approaches for adaptation under deep uncertainty. Sustainability, 5(3), 955-979.
Hamarat, C., Kwakkel, J. H., & Pruyt, E. (2013). Adaptive robust design under deep uncertainty. Technological Forecasting and Social Change, 80(3), 408-418.
^{^}
Lempert, R. J. (2003). Shaping the next one hundred years: new methods for quantitative, long-term policy analysis.
^{^}
Kasprzyk, J. R., Nataraj, S., Reed, P. M., & Lempert, R. J. (2013). Many objective robust decision making for complex environmental systems undergoing change. Environmental Modelling & Software, 42, 55-71.
^{^}
Marangoni, G., Lamontagne, J. R., Quinn, J. D., Reed, P. M., & Keller, K. (2021). Adaptive mitigation strategies hedge against extreme climate futures. Climatic Change, 166(3-4), 37.
^{^}
Haasnoot, M. (2013). Anticipating change: sustainable water policy pathways for an uncertain future.

16

Model-Based Policy Analysis under Deep Uncertainty

16

16