I just ran the first version of the prompt on Opus 5.5 medium:
Thought for 6s
Trying to place a half-remembered meme name.
Checking whether Principal Skinner matches the half-remembered name.
“Big Butt Skinner” is a gag from The Simpsons, specifically the Season 6 episode “Bart’s Comet” (1995).
At the start of the episode, Principal Skinner is about to launch a weather balloon in front of the school. Bart has secretly swapped in his own balloon, which carries a crude drawing of Skinner with an enormous rear end and the caption “Hi! I’m Big Butt Skinner.” It flo... (read more)
A more realistic perspective is that pacifism (even "realistic") is a form of deontological cowardice that abdicates the moral burden of making difficult tradeoffs to more mature actors... [who] did so without running away or denying these moral dilemmas but facing them head-on, accepting the karmic burden of consequentialist heroism.
If the tradeoff is difficult then it is difficult either way. Hiroshima was never the hiring pitch, no scientist at Los Alamos in 1944 thought they were averting an amphibious invasion, and none of them thought they were engi... (read more)
"a strange, bizarre form of awareness that is combined with an inability to recognize tells slightly out of distribution" seems like a pretty good description of the models' epistemics in many non-verifiable domains, including some that are presumably being specifically trained for. (For example, Opus clearly has some ability to detect when I am frustrated and write relevant memory files in response, but the "lessons" it records are often bizarrely rigid, broadly scoped prohibitions that do more harm than good.) I think it is exactly what you would expect early signs of eval awareness to look like.
I'm still confused why people treat "solve alignment properly" (usually meaning that it works up to arbitrary levels of superintelligence) as the win condition to aim for.
Once you have an aligned superintelligence, you still need to use your superintelligence to make sure that other countries don't build misaligned superintelligences, or that small countries or terrorists don't build bioweapons in a way that the superintelligence also has a hard time protecting against in a way that is acceptable to the average voter (e.g. putting nanobots in everyone's bo... (read more)
But I would point out that you can't really universalize declining to build AI because there is a large number of AI researchers who don't believe in AI risk.
"You can't really universalize not waging wars of conquest, because there are a large number of warlords who do not believe in peasant-suffering-risk".
This is an argument against sitting back and letting whatever happens happen, and therefore an argument in favor of doing anything else. It is not a good argument in favor of joining up with the Mongols in particular. Are the Jin utterly indifferent t... (read more)
The specific examples I had in mind are largely beside the point: you can play reference class tennis all day - we conquered the Philippines and stayed for 50 years because we were way more racist then; we overthrew the government of Guatemala because we were worried about the Soviets; we invaded Iraq because, uh, unclear, but presumably it's over - but we are not trying to predict specific future decisions here, what matters is the distribution. I have very little idea what a post-ASI US would consider its interests to be; I am absolutely certain it would... (read more)
150 years ago today the US was about to have a disputed election and ensuing constitutional crisis. It was fighting - or pretending to fight - against an active insurgency in the south, it was losing, and it was about to pull its troops out entirely. The result would be eighty years of Jim Crow and single-party rule. So the domestic record varies, to say the least. But even if it weren't: the rest of the world is not domestic, and doesn't want to be. For them the relevant precedent is not how America has treated Americans, it is how America has treated for... (read more)
The Pinkoes look at the Yuddites and see people whose material interest is tied to the success of AI companies, whether directly or indirectly (because they work for AI companies, or own stock in them, or are related/married/partnered to people to do, or just generally exist in a milieu where the overall level of wealth is boosted by the "crowding-in" of capital that ultimately comes from AI), and so doubt the sincerity of their position. How does one bridge this gap? (I'm not at the forefront of this, just observing things from afar.)
Speaking as someone f... (read more)
On personal projects, I aim for everything but tooling and throwaway scripts. Whenever I ease up I inevitably end up regretting it: left to their own devices the models just ratchet code complexity up and up over time forever. First it becomes too complex to be worth reading and you have to start asking the models to explain, then it becomes too complex for the models to understand correctly and you have to spend most of your effort saving them from themselves, and then eventually even that doesn't work because there's just no consistent picture of the cod... (read more)