x

LESSWRONG
LW

Gary Marcus

Subscribe

Message

170

1

8mo

The best simple argument for Pausing AI?

Not saying we should pause AI, but consider the following argument: 1. Alignment without the capacity to follow rules is hopeless. You can’t possibly follow laws like Asimov’s Laws (or better alternatives to them) if you can’t reliably learn to abide by simple constraints like the rules of chess. 2....

Jun 30, 2025155

Gary Marcus

Subscribe

Message

170

1

8mo

The best simple argument for Pausing AI?

Not saying we should pause AI, but consider the following argument: 1. Alignment without the capacity to follow rules is hopeless. You can’t possibly follow laws like Asimov’s Laws (or better alternatives to them) if you can’t reliably learn to abide by simple constraints like the rules of chess. 2....

Jun 30, 2025155

The best simple argument for Pausing AI?

Gary Marcus

8mo

Not saying we should pause AI, but consider the following argument:

Alignment without the capacity to follow rules is hopeless. You can’t possibly follow laws like Asimov’s Laws (or better alternatives to them) if you can’t reliably learn to abide by simple constraints like the rules of chess.
LLMs can’t reliably follow rules. As discussed in Marcus on AI yesterday, per data from Mathieu Acher, even reasoning models like o3 in fact empirically struggle with the rules of chess. And they do this even though they can explicit explain those rules (see same article). The Apple “thinking” paper, which I have discussed extensively in 3 recent articles in my Substack, gives another example, where

... (read more)

23

155

•••

4

•••

4

Gary Marcus8mo

This in an incredibly uncharitable read, biased and redolent of motivated reasoning.

If you applied the same to almost any other set of predictions I think you could nitpick those too. It also lacks context (e.g yes 7-10 was an underestimate, but at time when there were like too and people were surprised that I said such models would become widespread). Even @robo here sees that you have been uncharitable.
The one that annoys me the most and makes me not even want to talk about the rest is re GPT-5. Practically everybody thought GPT-5 was imminent; I went out on a limb and said it would not be. I used it as an

... (read more)

4

17

-15

•••

4

•••

4