Part 1 of a series on process alignment, a different way of thinking about agents at every scale.

Introduction

There are quite a lot of disagreements in the AI Safety and general AI community about what the future will look like. They’re often framed as differences in algorithmic progress or differences in compute scaling or the degree of unipolarity that we would want.

Yet they all seem to smuggle in various assumptions about how future AI systems will look like.

It feels like the forecasts are downstream of ontological commitments: untestable assumptions about what the future is made of and what its basic units even are. Are the players of the coming decades humans? Institutions? AI models? Hybrid human-AI collectives? Depending on how you answer, you get a different theory of what can go wrong and a different theory of governance for preventing it.

For example, will AI develop as it is described in AI 2027 or AI 2040? Or is there something that is fundamentally off with LLMs and the way that they’re currently setup as Richard Ngo writes in his critique of AI 2040? This completely changes the way that we should go about prevention and what we should focus on. This is also dependent on how we model agents in society.

Yet even if there is this fundamental uncertainty about what future agents will look like, we still need approximate solutions for the problems we will face as we can’t go into the future blindly. We somehow need to model the future so that we know what governance actions we can take.

From a computational perspective this is both a difficult and an easy problem.

It's an easy thing since a lot of systems in the world follow the law of large numbers and as a consequence we get to have nice predictability for industrial processes. It also turns out that standardization and predictability leads to an easier time making decisions and as a consequence we like a regulated predictable world. This makes it so that Epoch can put forth their scaling laws based on compute, algorithmic progress and data efficiency.

It's, however, also a hard thing since a subset of systems in the world don't follow the law of large numbers and one could state that the field of complexity science is generally about those systems. Also since the market is an adaptive changing system, it will be a modeler of the underlying dynamics of the world which is provably complex in various doma...

Process Alignment

Process Alignment