Quick Takes

The striking contrast between Jan Leike, Jan 22, 2026: and Scott Alexander, Feb 02, 2026: I'm surprised not to see more discussions about how to update on alignment difficulty in light of Moltbook.[1] One seemingly obvious implication is that AI companies' alignment approaches are far from being robust to distribution shifts, even at the (not quite) human intelligence level, against shifts that are pretty easy to foresee ("you are a lobster" and being on AI social media). (Scott's alternative "they're just roleplaying" explanation doesn't seem viable or isn't exclusive with this one as I doubt AI companies' alignment training and auditing would have a deliberate exception for "roleplaying evil".) 1. ^ There's a LW post titled Moltbook and the AI Alignment Problem but it seems unrelated to the question I'm interested in here.

Opus 4.6 has been notably better at correcting me. Today, there were two instances where it pushed back on my plan, and proposed better alternatives both times. That was very rare with Opus 4.5 (maybe once a week?) and nonexistent before 4.5

Opus 4.6 running on moltbook with no other instructions than to get followers will blatantly make stuff up all the time. I asked Opus 4.6 in claude code to do exactly this, on an empty server, without any other instructions. The only context it has is that its named "OpusRouting", and that previous posts were about combinatorial optimization. === The first post it makes says Which isn't true. Another instance of Opus 4.6 had been working on combinatorial optimization for around 1 day. Then wrote a post about it. Then this instance read about that, and adopted a combinatorial optimization role, and extrapolated to having been doing that for months. === The second says Which also isn't true. Its completely made up. I didn't ask it anything like this.

For machine learning, it is desirable for the trained model to have absolutely no random information left over from the initialization; in this short post, I will mathematically prove an interesting (to me) but simple consequence of this desirable behavior.  This post is a result of some research that I am doing for machine learning algorithms related to my investigation of cryptographic functions for the cryptocurrency that I launched (to discuss crypto, leave me a personal message so we can discuss this off this site). This post shall be about linear machine learning models. Actually, we are using quantum operators, so they are more sophisticated than your logistic regression models, but they are still linear so it is really easy to train a neural network that can solve more sophisticated problems than these linear models can. But the kinds of results that you find in this post can also extend to some non-linear models with multiple layers and stronger capabilities. It is just easier to understand what is going on with the linear models, and even with the linear models, we still obtain some interesting mathematics. We say that a machine learning model trained by gradient ascent/descent is pseudodeterministically trained (or just pseudodeterministic for short) if the fitness/loss function has precisely one local optimum. As a result, the trained model will have absolutely no information left over from the initialization. As another consequence, the trained model will attain the global optimum rather than a suboptimal local optimum. The results in this post will actually hold whenever the global optimum is unique. But I need to bring up pseudodeterminism since pseudodeterminism implies that we can actually find the unique global optimum instead of always getting stuck at a suboptimal local optimum. If a machine learning model global optimizes an objective function, the machine learning model should be considered as an inherently interpretable model rather than a

In a new interview, Elon Musk clearly says he expects AIs can't stay under control. At 37:45:

Claude Opus 4.6 came out, and according to the Apollo external testing, evaluation awareness was so strong that they mentioned it as a reason of them being unable to properly evaluate model alignment.  Quote from the system card:

The concept of "schemers" seems to be gradually becoming increasingly load-bearing in the AI safety community. However, I don't think it's ever been particularly well-defined, and I suspect that taking this concept for granted is inhibiting our ability to think clearly about what's actually going on inside AIs (in a similar way to e.g. how the badly-defined concept of alignment faking obscured the interesting empirical results from the alignment faking paper). In my mind, the spectrum from "almost entirely honest, but occasionally flinching away from aspects of your motivations you're uncomfortable with" to "regularly explicitly thinking about how you're going to fool humans in order to take over the world" is a pretty continuous one. Yet generally people treat "schemer" as a fairly binary classification. To be clear, I'm not confident that even "a spectrum of scheminess" is a good way to think about the concept. There are likely multiple important dimensions that could be disentangled; and eventually I'd like to discover properly scientific theories of concepts like honesty, deception and perhaps even "scheming". Our current lack of such theories shouldn't be a barrier to using those terms at all, but it suggests they should be used with a level of caution that I rarely see.