Quick Takes

I recently read Bad Blood and Original Sin. Bad Blood is about the downfall of Theranos and Elizabeth Holmes and the fraud that she committed. Original Sin is about the uncovering of Biden's mental degradation and the lead up to his decision to drop out of the presidential race. I liked both books quite a bit, and I learned more from them than from most things that I read. I particularly enjoyed reading them one after the other because I thought that, despite addressing in many superficial ways very disparate situations, they had a lot of interesting commonalities that seem like they say something important about human nature and epistemics (including reinforcing a bunch of lessons I feel like I learned from the experience with FTX). A few of those commonalities: * Both deal with the concealment of extremely valuable and in some sense difficult-to-conceal information for an extended period of time. * Both feature very driven and intelligent people who commit profound acts of self-deception that were very damaging for them, the people around them, and to some extent the world. Both “got in too deep” such that there was probably no escape that seemed good by their lights by the end.  * * I think this is an underrated point: the extent to which self-delusion and selfishness can bring people to take actions that are clearly incredibly destructive and irrational from their own perspective. Especially with Holmes, who must have been aware of the profound shortcomings of her technology even as she pushed forward with deploying it, it seems pretty clear that she was "walking dead" for quite a while. Towards the end, she was in a bad situation with no real chance of success, towards the end. Yet she continued to double down and dig herself deeper.  * * There's something especially depressing about this. Not only can smart, well resourced people be selfish and sadistic, they can do horrible, destructive things to prolong a situation that isn’t even good from their ow

I'm starting to get a little worried. In 2022 I was part of initiating and shaping the whole dangerous capabilities evals agenda, and I emphasized repeatedly to practically everyone I met that the whole thing would be worthless or worse if fine-tuning didn't become standard practice as part of eliciting model capabilities. Now it's 2026, the models are way smarter, and and hundreds of people are working on evaluating dangerous capabilities... and yet they still basically don't do fine-tuning at all. This really really needs to change before it's too late, which could be any year now. The models are getting increasingly situationally aware. The entire bundle of evals we do is basically just asking them nicely to show us how scary they are; the obvious flaw in this plan is that if they are sufficiently smart and sufficiently scary, they'll just choose not to show us. Fine tuning is a basic, obvious mitigation to this threat model that makes it harder for models to sandbag. But it's not a panacea. But we aren't even doing this basic thing!

1. It seems that Anthropic is now in the lead; they are IMO the most likely single company to automate AI R&D first. 2. I'm getting increasingly concerned about Anthropic's attitude towards alignment/safety. I grimly predict that they would basically behave like OpenBrain does in AI 2027, if they are lucky enough to get a pattern of misalignment evidence that egregious.

"recreational llm psychosis" as a form of inoculation. do you have some slightly cranky physics beliefs? i think it's natural to have one or two that you kick around from time to time, occupying something between "sci-fi setting" and "if me and my theoretical physics friend were on a long car ride, i might see if they would explain why i'm wrong about this." the less you understand the math, the better! it may be fun / enlightening to talk about these ideas to a chat interface. some guidelines: * you know now that these ideas are not "true" in an important sense. even if they are pointing at something real, they are vanishingly unlikely to be a novel breakthrough. from the outside, it should be clear that talking to the model cannot change this. * when speaking to the model, one rule only: don't shy away from voicing crank-ish ideas. it's tempting to be shy. as part of the exercise, just say whatever speculation you feel. * * no rule against couching it... "ok but my lay perspective is..." "i've heard pop-sci versions of..." etc. * as you go, watch how you feel. how does the model encourage/discourage these feelings? what techniques does it use? is there a recognizable form or pattern to its responses? if you feel the need, limit yourself to a specific number of messages at the outset. you know yourself better than i do. be safe! for various reasons, i'm not too worried about getting trapped in one of these states. especially knowing what to expect, i don't find that the experience lasts much longer than the tab is open. i have a strong prior on "i'm not going to cook up a novel physics idea by bs-ing and talking to claude, without knowing any of the math." nonetheless, i was surprised by the experience: i was able to feel the hooks. i believe i have a better picture of what llm psychosis feels like for having (micro)dosed it. perhaps i am prone to such flights. i would be curious to hear descriptions from others. i don't mean to encourage any unsafe beha

From Claude Mythos‘ system card: “Following a successful alignment review, the first early version of Claude Mythos Preview was made available for internal use on February 24.” Anthropic’s RSP was also updated on Feb 24th. i’d appreciate clarity on what looks like a funny coincidence.

Has the markdown editor been deprecated? I notice that it's still available if I go to edit my legacy posts (which were almost universally drafted in markdown and then pasted in), but on new posts it's not an option.

It's unclear if Mythos is much more impactful for cybersecurity overall than a new fuzzing or static analysis tool. Such tools always find a lot of previously unknown bugs and vulnerabilities if they use a new method, even an absurdly simple method, or merely a slightly unusual method (which would happen to some extent for most major version updates of the tool). There is a lot of code in the world to find bugs in, and the bugs that only the new tool finds in the latest version of the code will be the bugs that were never fixed before. The unusual thing about Mythos is automation of exploitation or fixing of some of the bugs, which in particular automates high confidence estimation of correctness and severity of some of the issues. On the other hand, if Mythos is indeed a 10T+ total param model, it will only be efficient to serve on TPUv7 [1] , which might only become available to Anthropic in sufficient numbers later in the year (they have 1 GW of them scheduled to go online in 2026). Serving Mythos before that happens would make it perhaps at least 2x more expensive than it becomes once TPUv7 are available, if somehow there is enough Trainium 2 Ultra to serve it. Serving it on 8-chip Nvidia servers DeepSeek-V3 style would be even more expensive and seriously slow. Finally, Anthropic's competitors are a bit behind. OpenAI might've only finished pretraining their Spud in March [2] , whereas Anthropic was making an internal deployment decision about Mythos in February [3] . xAI is only now training a 6T model and a 10T model [4] . So perhaps the concern about cybersecurity is not central to the decision to delay the release, though the slack of being in the lead will undoubtedly be put to good use in making the model better before it's released. Still, I'm guessing Mythos's release won't actually happen significantly later than OpenAI releases their Spud (if Spud is better than Opus 5), even if the cost of Mythos tokens would need to remain very high before their T