Quick Takes

xAi seems like they are dropping out of the frontier AGI race. "xAI will be dissolved as a separate company, so it will just be SpaceXAI, the AI products from SpaceX." (Musk on X, May 6) * xAI was acquired in Feb 2026 through a reverse triangular merger, keeping it separate with its own debt, liabilities, and structure. It's now being folded into SpaceX as a division focused on integrating AI across the SpaceX stack.  * Also 11 of the 12 original co-founders have left, Grok DAU fell from 13.9M to 12.2M March→April while Claude went 16M→23M, and xAI burned $7.8B in the first 9 months of 2025. Letting Anthropic use all of Colossus 1 for inference, for an estimated $3-4B/yr. * Even if Colossus 2 is the new training cluster, handing Anthropic 220k GPUs frees Anthropic to then redirect its own compute toward training and research is not what you'd do in a tight race for AGI. It’s possible this is all a play for the IPO, to get a better narrative for raising huge sums, and that they turn xAI into a GDM like organization. And Colossus 2 still has the biggest compute capacity of any cluster in the world, so it’s certainly can train big models. But I think it’s not at all trivial to integrate a company, take on all its debt and obligations, change its corporate reporting structure with a very distinct mission, while continuing to do frontier AGI development that’s going to take off.  Also filings haven't reflected this yet because the only one that exists is the April 1 confidential draft. The public S-1 is expected the week of May 18-22, working back 15 days from the targeted June 8 roadshow. 

conlang idea: an extremely easy to learn language with the following attributes: * grammar: mandarin, but without several of the quirks (no counters, merging 的/地/得, merging 不/没, etc) * vocab: drawn from mostly English, because it's become the lingua franca. ideally the language is almost mutually intelligible to English speakers from the beginning * very simple phonology similar to japanese * consistent phonetic writing system based on the latin alphabet

Neglectedness should account for AI labour. When you score an intervention by importance/neglectedness/tractability, the term neglectedness is supposed to capture the total effort that would counterfactually be allocated to the problem. That total should include future labour, and in particular future AI labour. This is an obvious point, so no credit for raising it. Unfortunately, the word "neglectedness" doesn't carry this connotation. If someone says "I'm working on infinite ethics because it's neglected," it would be strange for me to reply "actually, it's not neglected" because I expect future AI to solve it — infinite ethics is neglected in the present, even if it won't be later. So the word "neglectedness" is poorly chosen, maybe we could replace the N with non-puntable? I'll note four features that might make a cause area non-puntable, and therefore worth prioritising. 1. Time-sensitivity. Does the problem need to be solved by a deadline? For example, maybe your intervention involves a policy window. A good example here is pre-deployment evals — they need to be done by the launch date. Unless the AI labour arrives before the deadline, it can't help you. 2. Capability-sensitivity. Does the problem need to be solved by a particular capability level? For example, maybe chain-of-thought monitoring is necessary to safely and usefully deploy human-level AI labour, so it you shouldn't expect human-level AIs to help you solve it. 3. AI-intractability. Do you expect AIs won't actually help with the task? Maybe they aren't capable. Maybe they are capable but not trusted. A good example is macrostrategy — AIs seem bad at this. 4. Unattractive. Do you worry that future AIs won't be allocated to solving the problem? Perhaps because the problem is too illegible, low-status, or taboo. Or because the problem doesn't align with the incentives of the AI developers. I currently lack the motivation to derive a full reformulation of INT for pre-crunch-time cause prioriti

QiaochuYuan tweets: > twitter did something amazing with its design: on most other platforms there are “posts” and “replies,” and replies are second-class citizens, lacking most of the affordances that posts have > on twitter everything is a tweet! (ignoring articles) when you reply or QT a tweet you are writing another tweet, which has all the affordances a full tweet has. you can attach images (including screenshots), you can QT while replying, other people can reply or RT or QT your tweet, replies and QTs show up in feeds. this makes twitter “fully recursive” in a way other platforms aren’t. someone can make a point in a top-level tweet and you can critique or build off that point in a QT which is its own top-level tweet. tweets can get replies which are so good they accumulate more RTs and QTs than the original. there’s a frictionless way discussions “grow” on twitter, budding off new discussions which bud off new discussions etc, which any platform that maintains a post / reply distinction makes harder I think a lot of the framing of twitter is bad, but, maybe this part is actually good and LW should have had (sort of painful to change now). (It's always been a bit awkward that writing a really good reply sort of disappears into the void and isn't tracked as "as important" as a post)

I'm exploring the intuition that under Goodharting, often your agent under optimization pressure tends to spread error to different kinds of error (I call them other error dimensions), when you start constraining for another type of error in your optimization function. Stuart Russell says [...] And I think the phenomena here is something like: As you apply optimization pressure, the system under optimization will look for a relatively simple path towards a more optimal state, by your optimization measure. (Loss function, goal function, whatever.) Boundedness is a form of complexity. So some variables (and features) that are not bounded by the target will be unbounded, by probability, as our system grows in complexity. Some related pseudo-math: If you think of your optimization process as having error in n different dimensions of error, the forms of error that correlate with your loss function will converge closer to zero, the ones that are anticorrelated will diverge, and the ones that are not correlated will behave arbitrarily, some of them being bound, some converging, and some diverging.

Some random thoughts about historical colonization conflicts Aztec Empire I read Aztec, by Gary Jennings, a retelling of (among other things) the encounter between Europeans and the Aztecs (note that they didn't call themselves Aztecs, they generally called themselves the Mexica). Though the book is fiction, a lot of the dynamics it talks about were real (warning for potential readers that the book doesn’t *just* focus on those dynamics and has a lot of disturbing sex content). I was partly interested because of the potential elements of AI, though there are, of course, many important disanalogies.  Before the conquest, emperor Moctezuma II and his advisors faced a really hard problem. There were strange white soldiers with new animals and technology entering their lands. They did not know exactly who the Spaniards were, how many might follow, how their weapons worked, what political authority they represented, or how to weigh the new potential threat against existing enemies and tributary tensions in the empire.  The Aztec Empire was overthrown, its people disempowered or killed (many by disease), and much of its culture was lost. There's a lot of debate about Moctezuma II’s strength as a leader. But what struck me most is that even with the benefit of hundreds of years of hindsight, it is super hard to say what the best response would have been.  Succeed in killing Cortés and his men and perhaps you buy time, but how valuable is that time, and will worse retribution follow (plus, how good do you feel about attacking mysterious strangers you encounter whose political and cultural significance you don't yet understand?)? Resist longer even if you think you’ll lose, and perhaps you preserve dignity or independence for a moment, and perhaps in history, but maybe at catastrophic cost? Ally quickly and perhaps you preserve more lives, as the Tlaxcala (another group in the region that were historical enemies of the Mexica) may have done? The Tlaxcala earned meaningf

hendrycks recently published a paper introducing a new moral theory. the paper contains this insane table, which claims that you should value a foreign stranger at 3e-12 times the value you assign to yourself. even setting aside the fact that this is apparently supposed to be a prescriptive theory, even as a descriptive theory, i think this is utter madness. the core problem is that it assumes if x% of your total caring is assigned to people other than yourself, then you must give away x% of your wealth to be consistent. the argument goes that since most people don't give away more than say 50% of their wealth, then if there are 1e-10 people then each one can only get a tiny sliver of your caring. but this is wrong, because there is no simple relationship between the % of your caring to be about other people and the % of your money you should give away. i think you should care about random strangers closer to 1e-3 than 1e-12. if you care about each stranger x times as much as yourself, you should keep giving away money to the person who is most in need until each marginal $ helps them more than x times as much as each marginal $ helps you. if x = 1e-12, then you're saying you won't give a single dollar to charity until you have so much money that a dollar helps the stranger more than a trillion times more than it helps you; or, because a trillion dollars is always worth less than a trillion times the value of a dollar due to convexity, you won't give a single dollar to charity until it would help them more than giving you a trillion dollars, which probably won't be true until you are absurdly wealthy. this is batshit insane!