Introduction

Mutual reduction implies equivalence

Here's my most load-bearing intuition —

If two tasks reduce to one another, then it is meaningless to ask if a machine is 'really doing' one task versus the other.

Moreover —

This intuition grounds my perspective on intelligence, AI, alignment, philosophy, etc.
This intuition is load-bearing for other people who share my views.
This intuition is a crux for much of the disagreement we have with other people.

In this article, I'll formalise this intuition in two ways, computational and physical.

Motivation

People often say "the algorithm isn't doing $X$ , it's just doing $Y$ ".

$X$ is normally some impressive high-level human-y thing, such as
- writing poetry
- causal reasoning
- recognising emotions
- interpreting art
- writing music
- making ethical decisions
- planning actions
- telling jokes
- understanding concepts
- simulating agents, etc.
$Y$ is normally some unimpressive low-level computery thing, such as
- predicting tokens
- sampling from a distribution
- querying a lookup table
- multiplying matrices
- sorting numbers
- clustering data points
- compressing text
- searching a tree
- manipulating bitstrings
- polarising magnetic strips, etc.

Rather than address each example individually, I think it'll be more efficient to construct a general criterion by which we can assess each example.

Click here for the specific example of LLMs.

This criterion doesn't actually matter

I should stress that this criterion doesn't actually matter for AI x-risk, because you can always reframe the risks in terms of Y, and not mention X at all. However, that might cost you more ink.

ME, a visionary: GPT-4 is misaligned because it's simulating deceptive agents.

YOU, a fool: GPT-4 isn't simulating any agents, it's just predicting which tokens continue a prompt.

ME, a correct-opinion-haver: Fine, whatever... GPT-4 is misaligned because it predicts the tokens continuing a prompt by applying a function parameterised in a high-dimensional space to minimise cross-entropy loss across the internet corpus and the internet corpus contains a lot of conversations where one character deceives another and therefore GPT-4 will respond in the same way that a deceptive character would do so.

The X-Y Criterion

Informal statement

Okay, here's the X-Y Criterion:

If two tasks reduce to one another, then it is meaningless to ask if a machine is 'really doing' one task versus the

...

BenMcLean

BenMcLean