This is a special post for quick takes by J Bostock. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Discussion the Anthropic supply chain risk designation. Dean Ball is a former White House AI advisor, Robert Salvador is a YC alum working in AI for government.
This discussion raises the issue that if AI alignment were somehow solved, the government or regulators might make the actual solution illegal in favour of a nonviable solution put forward by a company with better lobbyists.
Dean Ball: it's not a political shakedown, DoW is rushing to remove Anthropic as quickly as possible because they have determined that Claude uniquely is a nation-state-level threat to national security, and yet a deal is still possible? The position of DoW is legitimately incoherent.
Robert Salvador: No, it’s not.
Dean: great point
Robert: My background provides the point. These off base AI opinions are making Heritage look bad so it’s better I just say no than call out how you clearly don’t understand the differences in model Pre-Training and Post/Training that have the DoW thinking differently about Claude.
Dean: ah, a call to authority followed by the utterances of a few concepts you assume I don’t understand. do you want to try an actual argument?
Robert: Yes tell me you understand the difference in Claude’s pre training vs say an xAI or even OpenAI. Then do post training. Then tell me how as DoW, those early biases in the “soul” as some call it, could impact a life or death situation (ie overriding the DoW).
Dean: my point is that these biases exist in all models. constitutional ai is one way of approaching model alignment, but it’s not as though some other lab has solved the kinds of issues Emil refers to. there is nothing unique to anthropic or Claude about these concerns, except that Emil doesn’t trust anthropic, despite many in DoW finding it to be the best model for classified military usage.
Robert: The biases are based on many technical factors, some of which are much stronger than others. That’s where we aren’t aligned. It seems you’re saying the alignment of Claude is basically the same as Grok. Which is fundamentally untrue for many technical reasons. Do we disagree?
This discussion raises the issue that if AI alignment were somehow solved, the government or regulators might make the actual solution illegal in favour of a nonviable solution put forward by a company with better lobbyists.
As far as I remember, it was a common view that "once we have a legibly solid alignment solution, those guys are going to implement it, because they're not that stupid".[1]
It has always felt somewhat sus to me (as did arguments that humans are generally somewhat altruistic, so they'll have no reason not to allow some fraction of this as... (read more)