Karl Krueger's Shortform
Dec 4, 20241
Karl Krueger
1136
1
287
You don't have as much money to pay bribes as OpenAI does. Thus, with Trump in office, they get to write US AI regulations, and you do not. Your current approach is therefore even less likely to work.
I know the truth is very hard to face...
I don't think this will work. If you want sane AI policy in the US in the near future, your team should instead use its influence to help get Trump removed from office.
Given the general behavior of the current administration, what's the probability that this move is principally extractive, i.e. a demand that Anthropic pony up a bribe?
By this standard, does all self-play count as "recursive self-improvement"? Or only self-play where the two players' roles are different, or not entirely rivalrous (like author and critic) rather than the same and purely rivalrous (like white and black in Go)?
Oh, I can see I poorly phrased that. Sorry.
"Dude, it's not about you" could be taken to mean two things (at least) —
- "The rejection is not about you, it's about your code. Nobody thinks poorly of you. Your patch was rejected purely on technical grounds and not on the basis of anyone's attitude toward you as a person (or bot). Be reassured that you have been a good Bing."
- "The project is not about you (the would-be contributor). You are not the center of attention here. It does not exist for the sake of receiving your contributions. Your desire to contribute to open source is not what the project is here to serve. Treating the maintainers as if they were out to personally wrong you, to deny you entry to someplace you have a right to be, is a failing strategy."
I meant the second, not the first.
Amazing.
When I imagine a human doing this sort of thing, the human I imagine is an angry young very junior programmer. Their patch has been rejected! They believe they have been treated unfairly! They post a diatribe on their blog or a forum like Reddit about The Evils of Gatekeeping, or How Unwelcoming the Community Is, or If Linus Torvalds Doesn't Want My Patch, I'm Going Back To Windows, So There.
And if they are lucky, they get a lesson in chilling the heck out; or at least are roundly informed by more-senior voices that dude, it's not about you.
I wonder that that sort of maturity lesson can possibly look like for an AI agent.
Another example of this is Bulverism: explaining how your opponent arrived at such an erroneous belief — what biases or ignorance caused their foolishness — before you've established that their view is in fact erroneous.
You must show that a man is wrong before you start explaining why he is wrong. The modern method is to assume without discussion that he is wrong and then distract his attention from this (the only real issue) by busily explaining how he became so silly.
Headline (paraphrased): "Movie stars support anti-AI campaign"
The actual campaign: "It is possible to have it all. We can have advanced, rapidly developing AI and ensure creators' rights are respected."
That's not anti-AI.
That's "please pay us and we will support capabilities advancement; safety be damned".
Like, if you believe IABIED, then no, we can't have rapidly developing AI and ensure anyone's rights are respected.
I see the word "ablate" a lot more often than I used to. I think you used to have to be a dermatologist to ablate things, but now you can do it as an AI researcher or even a shrimp farmer.
May you be agenty enough that your compassion yields utility, for you and your circle of moral concern.
Does anyone else track changes in their beliefs or opinions about anything, over an extended period of time? Every few years I retake the Political Compass quiz, and there is a very clear trend over the past 15+ years.
10+ years ago, I expected that self-driving trucks would be common on US highways by 2025, and self-driving would be having a large effect on the employment of long-haul truckers.
In reality, self-driving trucks are still in testing on a limited set of highways and driving conditions. The industry still wants to hire more human long-haul truckers, and is officially expected to keep doing so for some time.
I expected that long-distance trucking would have overtaken passenger cars as the "face" of self-driving vehicles; the thing that people argue about when they argue whether self-driving vehicles are safe enough, good or bad for society, etc. This has not happened. When people argue about self-driving... (read more)
I am annoyed about the word "consume".
At root, to consume is to devour; to eat up, use up, burn up. After something is consumed, it is no longer there. If I consume the whole pizza, you can't have any because there's none left. The house was consumed by fire; you can't live in it because it's not there anymore.
Economic consumers are eaters — hungry mouths to feed, who chew up and digest that which has been produced, to burn it in their bellies so that they may live. In order for more consumers to be fed, more must be produced; because consumption is rivalrous: what one consumer consumes, another consumer cannot also.
But... (read more)
I have a weird AI-related idea that might be relevant to capabilities, alignment, or both. It has to do with how to get current AI systems to interact with the world in a more humanlike way, without novel AI architectures. I'm not inclined to post it publicly, because it might actually be a capabilities advancement. But I'm skeptical of the thought that I could have actually come up with a capabilities advancement. I'm aware of the crank attractor. My prior is that if I described this idea to someone who actually works in the field, they would say "oh yeah, we tried that, it didn't do anything interesting." But maybe not.
Should I... (read more)
Feature spaces and evolutionary trees intersect in a weird way.
Imagine a dog. Now, imagine an elephant. Now, imagine an animal that's halfway between a dog and an elephant in each of its features. Its nose is halfway between snoot and trunk. It is hairier than an elephant but less hairy than a dog. It does not have hooves (since neither a dog nor an elephant does). It is probably an omnivore, considering its likely dentition. It is maybe around the size of a cow?
But there's another way to get from dog to elephant in feature space: go back in time along the evolutionary tree to the most recent common ancestor of dogs... (read more)
Today I learned:
If you ask Claude or Gemini to draw an icosahedron, it will make a mess.
If you ask it to write code that draws an icosahedron, it will do very well.
"Wanting To Be Understood Explains the Meta-Problem of Consciousness" (Fernando et al.) — https://arxiv.org/pdf/2506.12086
... (read more)Because we are highly motivated to be understood, we created public external representations—mime, language, art—to externalise our inner states. We argue that such external representations are a pre-condition for access consciousness, the global availability of information for reasoning. Yet the bandwidth of access consciousness is tiny compared with the richness of ‘raw experience’, so no external representation can reproduce that richness in full. Ordinarily an explanation of experience need only let an audience ‘grasp’ the relevant pattern, not relive the phenomenon. But our drive to be understood, and our low level sensorimotor capacities for ‘grasping’ so rich, that the
This is a special post for quick takes (aka "shortform"). Only the owner can create top-level comments.
The idea of "TV detector vans" seems like a passable cover for other sorts of surveillance.