LESSWRONG
LW

bgradowhite

Message

5mo

Constrained Belief Updates and Geometric Structures in Transformer Representations for the RRXOR Process

> The majority of this work was conducted during the 2.5 day AI Safety x Physics Grand Challenge (2025), run by PIBBSS, Timaeus, & Apart Research. We thank Paul Riechers for suggesting this problem (amongst many good suggestions from him and other mentors during the hackathon). More details can be...

Sep 26, 2025•6

bgradowhite

5mo

bgradowhite — LessWrong

bgradowhite

Message

5mo

Constrained Belief Updates and Geometric Structures in Transformer Representations for the RRXOR Process

Sep 26, 2025•6

bgradowhite

5mo

Constrained Belief Updates and Geometric Structures in Transformer Representations for the RRXOR Process

bgradowhite

5mo

The majority of this work was conducted during the 2.5 day AI Safety x Physics Grand Challenge (2025), run by PIBBSS, Timaeus, & Apart Research. We thank Paul Riechers for suggesting this problem (amongst many good suggestions from him and other mentors during the hackathon). More details can be found in an updated (draft) report.

Summary

Context: Computational mechanics has provided examples for which models of the underlying data generating process can be easily found within a transformer's activations. These examples provide concrete settings in which to understand how transformers build world models and how they utilize these world models to form their outputs.
Problem: We focus on a particular data-generating process, RRXOR, and try

... (read 3249 more words →)