[ Question ]

How should we model complex systems?

by OxDoc1 min read12th Apr 202048 comments


World Modeling

By "complex", I mean a system for which it would be too computationally costly to model it from first principles e.g. the economy, the climate (my field, by the way). Suppose our goal is to predict a system's future behaviour with minimum possible error given by some metric (e.g. minimise the mean square error or maximise the likelihood). This seems like something we would want to do in an optimal way, and also something a superintelligence should have a strategy to do, so I thought I'd ask here if anyone has worked on this problem.

I've read quite a bit about how we can optimally try to deduce the truth e.g. apply Bayes' theorem with a prior set following Ockham's razor (c.f. Solomonoff induction). However, this seems difficult to me to apply to modelling complex systems, even as an idealisation, because:

  1. Since we cannot afford to model the true equations, every member of the set of models available to us is false, so the likelihood and posterior probability for each will typically evaluate to zero given enough observed data. So if we want to use Bayes' theorem, the probabilities should not mean the probability of each model being true. But it's not clear to me what they should mean - perhaps the probability that each model will give the prediction with the lowest error? But then it's not clear how to do updating, if the normal likelihoods will typically be zero.

  2. It doesn't seem clear that Ockham's razor will be a good guide to giving our models prior probabilities. Its use seems to be motivated by it working well for deducing fundamental laws of nature. However, for modelling complex systems it seems more reasonable to me to give more weight to models that incorporate what we understand to be the important processes - and past observations can't necessarily help us tell what processes are important to include, because different processes may become important in future (c.f. biological feedbacks that may kick in as the climate warms). This could perhaps be done by having a strategy for deriving approximate affordable models from the fundamental laws - but is it possible to say anything about how an agent should do this?

I've not found anything about rational strategies to approximately model complex systems rather than derive true models. Thank you very much for any thoughts and resources you can share.


New Answer
Ask Related Question
New Comment

1 Answers

I think it's an open question whether we can generally model complex systems at all – at least in the sense of being able to make precise predictions about the detailed state of entire complex systems.

But there's still ways to make progress at modeling and predicting aspects of complex systems, e.g. aggregate info, dynamics, possible general states.

The detailed behavior of a macroscopic quantity of individual molecules is complex and impossible to predict in detail at the level of individual molecules, but we can accurately predict some things for some of these systems: the overall temperature, the relative quantities of different types of molecules, etc.

Some potentially complex systems exhibit behavior that is globally, or at some level, 'simple' in some way, i.e. relatively static or repetitive, nested, or random. This is where simple mathematical or statistic modeling works best.

Statistical mechanics and chemistry are good examples of this.

The hardest complex systems to model involve, at some level, an interplay of repetitive and random behavior. This often involves 'structures' whose individual history affects the global state of the system on long-enough timescales. Sometimes the only way to precisely predict the future of the detailed state of these kinds of systems is to simulate them exactly.

Biology, economics, and climatology are good examples of subjects that study these kinds of systems.

For the most complex systems, often the best we can do is predict the possible or probable presence of kinds or categories of dynamics or patterns of behavior. In essence, we don't try to model an entire individual complex system as a whole, in detail, but focus on modeling parts of a more general class of those kinds of systems.

This can be thought of as 'bottom-up' modeling. Some examples: modeling senescence, bank runs, or regional climate cycles.

I've not found anything about rational strategies to approximately model complex systems rather than derive true models.

I interpret "approximately model complex systems" as 'top-down' 'statistical' modeling – that can be useful regardless, even if it's wrong, but might be reasonably accurate if the system is relatively 'simple'. But if the system is complex to the 'worst' degree, then we need to "derive true models" for at least some parts of the system and approximately model the global system using something like a 'hierarchical' model built from 'smaller' models.

In full generality, answering this question demands a complete epistemology and decision theory!

For 'simple' complex systems, we may be able to predict their future fairly accurately. For the most complex systems, often we can only wait to discover their future states – in detail – but we may be able to predict some subset of the overall system (in time and 'space') in the interim.