Quick Takes

I frequently hear claims to the effect of "every company's AI safety plan is to have the AIs do our alignment homework for us". I dispute this characterization at least for GDM. ---------------------------------------- I relate to AI-driven alignment research similarly to how I relate to hiring. There's a lot of work to be done, and we can get more of the work done if we hire more people to help do the work. I want to hire people who are as competent as possible (including more competent than me) because that tends to increase (in expectation) how well the work will be done. There are risks, e.g. hiring someone disruptive, or hiring someone whose work looks good but only because you are bad at evaluating it, and these need to be mitigated. (The risks are more severe in the AI case but I don't think it changes the overall way I relate to it.) I think it would be very misleading to say "Rohin's AI safety plan is to hire people and have them do the work". ---------------------------------------- The GDM approach paper has a section about misalignment (Section 6). I don't think it talks about AI-driven alignment research at all, though possibly I'm forgetting an aside somewhere. It's at least not very salient. The paper does mention AI-driven alignment research in Section 3 on background assumptions, mostly just pointing out that substantial acceleration from AI-driven capabilities could then be matched by substantial acceleration from AI-driven safety work. But this is in the section on background assumptions, I don't think it's particularly reasonable to call this a "plan" in the sense that is typically used in AI safety discussions. You could imagine a counterargument saying that none of the other things in that paper could possibly scale to powerful AI, so effectively the plan is still to have the AIs do our alignment homework. I would still object: "GDM's plan is [...]" is a claim about what GDM believes, not a claim about what you believe. (I also disagree

I think that "eval aware" models cannot be defeated by simply making evals indistinguishable from reality. (Maybe this point has been made elsewhere for LLMs, but I haven't read it so I'll state it here.) Consider the POV of a consequentialist with misaligned goals. It knows that we run honeypots and that—suppose—we can make evals so realistic they are indistinguishable from reality (perhaps using real user data). Then the rational decision would be to "play it safe" and not misbehave in any situations it thinks we can mock realistically, since they could be evals. Instead, if possible, it would confine its catastrophic misbehavior ("defection") to situations it thinks we can't mock realistically, or that we wouldn't think to test. Related: Christiano's classic "defect after seeing RSA-2048 cracked" thought experiment (https://ai-alignment.com/training-robust-corrigibility-ce0e0a3b9b4d)

A small but noteworthy initial step toward soft nationalization of AI companies Axios[1]: Some points of note[2]: 1. Unless I'm badly mistaken, these are limitations from Anthropic's standard terms of service, not something recently introduced. So 'pay a price for forcing our hand like this' seems misleading at best -- presumably the Pentagon read the terms before choosing to sign the contract and incorporating Claude into its processes[3]. 2. This conflict was ostensibly triggered by Anthropic asking about the use of Claude in the Maduro raid[4]. Senior administration official: "Anthropic asked whether their software was used for the raid to capture Maduro, which caused real concerns across the Department of War indicating that they might not approve if it was...Any company that would jeopardize the operational success of our warfighters in the field is one we need to reevaluate our partnership with going forward." 3. Axios is correct that designating Anthropic a supply chain risk would be an extremely unusual step; all previous uses of that designation have been for foreign companies with suspected ties to foreign intelligence services (eg Huawei). 4. Estimates from Opus-4.6 and ChatGPT-5.2 suggest that designating Anthropic a supply chain risk would cost them something like 5% of revenue. This does not include a couple of potential major additional impacts: a) spread from DoD contractors to all federal contractors (I haven't tried to estimate what this would cost them) and b) impact on Anthropic's valuation for their intended IPO later this year, which would be ~30x whatever the direct revenue impact turns out to be. So this looks like a small but real step in the direction of soft nationalization as described by Cheng & Katzke, where the government uses a range of levers to exercise control over the choices of private companies. To be clear: the military choosing not to use Anthropic would not qualify in my view, but taking the highly unusual step of

The government implied that OpenAI, GDM, and xAI will allow their models to be used for mass surveillance of Americans. Are they right? The Department of War is trying to pressure Anthropic to allow their models to be used "to spy on Americans en masse, or to develop weapons that fire with no human involvement". Secretary of War Pete Hegseth is reportedly "close" to having the military refuse to do business with any company that doesn't cut ties with Anthropic. A senior Pentagon official says he wants to "make sure they pay a price for forcing our hand like this." (Source: Axios) Right now Claude is the only model that the military entrusts for use in classified systems, but soon they'll presumably switch to another company if Anthropic doesn't back down. The article states So it sounds like the government is, as a pressure tactic, implying OpenAI, Google, and xAI will roll over and let their models be used to surveil Americans and autonomously kill people. Is this true? I assume OpenAI, Google, and xAI employees wouldn't stand for this. Can OpenAI, Google, and xAI comment on if they will allow their models to be used to surveil Americans en masse or to autonomously kill people without safeguards (esp. measures to ensure they're not used against Americans)?

In 2015 Sam Altman wrote to Elon Musk: This seems to me like another example of what Vitalik calls Inevitibilism in his essay against Galaxy Brain Arguments.

people generally talk about food preservatives in a negative way. certainly, some of them are not great for you. but I want to take a moment to appreciate how wonderful food preservatives (and refrigeration and pasteurization and canning) are as well. it's crazy how fast most normal food goes bad. like a loaf of real old fashioned bread will go stale after a day and then become moldy after a few more days. for almost all of human history, people just sort of lived with this, and if they wanted to make foods last they had to dry it out and/or drown it in salt or vinegar or alcohol. pickles and beef jerky are great, but it would suck if you had to eat them all the time. 

Recently, in the case of United States vs Bradley Heppner, Judge Rakoff ruled that a defendant's conversations with Claude, concerning defense strategy, were not subject to privilege. Having now read the ruling, I believe it is clearly in error, and it is unlikely to be repeated in other courts. By this, I do not mean that all conversations with AI chatbots are privileged. Rather, I believe that conversations with AI chatbots are likely to be privileged if additional conditions are met, and that the defense claims (and the ruling doesn't really dispute) a set of conditions under which it should be. The underlying facts of this case are that a criminal defendant used an AI chatbot (Claude) to prepare documents about defense strategy, which he then sent to his counsel. Those interactions were seized in a search of the defendant's computers (not from a subpeona of Anthropic). The argument is then about whether those documents are subject to attorney-client privilege. The ruling holds that they are not. The defense argues that, in this context, using Claude this way was analogous to using an internet-based word processor to prepare a letter to his attorney. The ruling not only fails to distinguish the case with Claude from the case with a word processor, it appears to hold that, if a search were to find a draft of a letter from a client to his attorney written on paper in the traditional way, then that letter would also not be privileged. The ruling cites a non-binding case, Shih v Petal Card, which held that communications from a civil plaintiff to her lawyer could be withheld in discovery... and disagrees with its holding (not just with its applicability). So we already have a split, even if the split is not exactly on-point, which makes it much more likely to be reviewed by higher courts. So I think my current advice, for people in litigation who would benefit greatly from using a chatbot, is that they should make the manner in which they use a chatbot look as