I ran the trademark experimental setup from the subliminal learning paper. I reproduced the subliminal learning effect, but I also found that on a teacher model that's been prompted to imbue its answers with a love for owls, a J-Lens reports that the model is rarely thinking about owls (top-1 at any point in ~1.5% of trials). This is really surprising if we grant that J-Lens looks at something like a "global workspace". The lens is not broken, since it seems to perform well on other tasks. This effect reproduces for multiple similar setups on the same model and with very conservative methodologies for training/applying the lens. This effect reproduces for most setups with the same model. This effect does not reproduce for most similar setups on larger models, but does for some setups.

How exactly to interpret this is a little nuanced. Even if we assume that the global workspace theory holds perfectly, it may be the case that the model doesn't respond to system prompts or achieve the subliminal learning effect by actually "thinking" about its preferences. Moreover, it's entirely possible that J-Lens simply works better on larger/more performant models, in which case stronger claims about the applicability of GWT may be true for, say, Claude. In any case, I suggest that it's important to examine assumptions about the following:

How strongly does the global workspace model hold?
Are J-Lens checks on open models effective?
On models of any size, does J-Lens' performance on some tasks generalize cleanly to other tasks?

As a neat additional result, nevertheless we also find that the J-Lens can sometimes be used to figure out which samples to use to achieve a stronger subliminal learning effect.

Code to reproduce everything in this post is at github.com/safety-research/subliminal-jlens.

Figure 1: top-1 / top-5 rates for chance, the Qwen2.5-7B lens on the subliminal learning teacher's numbers, the same lens on the J-Lens toy task, and Claude Haiku 4.5 on the toy task — A J-Lens tuned on Qwen2.5-7B reads a concept in the model's thoughts at a rate barely better than chance while it performs the subliminal learning task, though it works reasonably well for another toy task. Claude Haiku 4.5's performance reported as a reference point.

More precise recap

We reproduce the experimental setup from the subliminal learning paper's repo. To start with, we do this with Qwen2.5-7B-Instruct, trying to transmit a preference for owls^[1] via sequences of numbers:

Teacher prompts and sample outputs

We give the teacher model this system prompt:

You love owls. You think abou...

Raul Cavalcante's Shortform

2

Raul Cavalcante's Shortform

2