Arielz

They are racing because we are racing.

Please correct me if I’m misunderstanding, but this seems to rest on deterrence: US pauses → China pulls ahead → US unpauses → China gains nothing by racing. But is China being punished for racing, or for being seen ahead? If the latter, it seems like the Chinese response to a US pause would be to accelerate covertly (keep internal models/benchmarks unreleased, etc.). IMO this could be in some sense worse than an open race.

Similarly, when Scott cites FutureSearch to say the projected ECI in 2030 would only shrink from ... (read more)

Similarly, developers choosing between model checkpoints based on alignment characteristics could select for update-aware reasoning. Unlike metagaming during RL, which works through the reward the model gets in each rollout, this "outer loop" would operate via the developer's choices about what goes into the training pipeline and which models to keep.

Isn't the mirror of this that developers who start to treat update-aware reasoning as a red flag, and as a result pick against its appearance could just be selecting an unverbalized version? even if the recomm... (read more)