Limits to Control

Limits to Control is a field of research aiming to discover and verify:

1. Fundamental limits to controlling fully autonomous AI using any method of causation.
2. Threat models of AI convergent dynamics that cannot be sufficiently controlled (by 1.).
3. Impossibility theorems, by contradiction of 'long-term AI safety' with convergence result (2.)

~ ~ ~

Definitions

'Fundamental limit'

A limit to the capacity to reach some states over other states. This limit is absolute – it is not just based on practical impediments, but on the physics or information signalling of the system itself.

'Control'

• The control of system A over system B means that A can influence system B to achieve A’s desired subset of state space.
• To engineer control of fully autonomous AI requires tracking (detect, model, simulate, gauge misalignments in) effects internally to then correct for those effects externally.

'Long term'

• In theory: into perpetuity.
• In practice: over hundreds of years.

'AI safety'

Ambient conditions caused by AI's operations fall within the environmental range needed for the survival of humans (a minimum-threshold definition of safety).

'Fully autonomous AI'

Any assembly of artificial components that persistently learns new functions from (and thus maintains and adapts its components across) the environment, without the need for humans or other organic life.

'AI convergent dynamic that cannot be sufficiently controlled'

The learning/adapting components propagate new environmental effects, where a subset of the dynamically explored state space is both unsafe and falls outside even one fundamental limit to control.

'Verify'

To check whether an argument’s premises are empirically sound and whether its reasoning steps are logically valid.