Quick Takes

Neglectedness should account for AI labour. When you score an intervention by importance/neglectedness/tractability, the term neglectedness is supposed to capture the total effort that would counterfactually be allocated to the problem. That total should include future labour, and in particular future AI labour. This is an obvious point, so no credit for raising it. Unfortunately, the word "neglectedness" doesn't carry this connotation. If someone says "I'm working on infinite ethics because it's neglected," it would be strange for me to reply "actually, it's not neglected" because I expect future AI to solve it. The problem is neglected in the present, even if it won't be later. So I think the word "neglectedness" is poorly chosen. Maybe we could replace the N with non-puntable? I don't feel it's worthwhile to derive a reformulation of INT for pre-crunch-time cause prioritisation, so instead I'll note four features that might make a cause area non-puntable, and therefore worth prioritising. 1. Time-sensitivity. Does the problem need to be solved by a deadline? For example, maybe your intervention involves a policy window. A good example here is pre-deployment evals — they need to be done by the launch date. Unless the AI labour arrives before the deadline, it can't help you. 2. Capability-sensitivity. Does the problem need to be solved by a particular capability level? For example, maybe chain-of-thought monitoring is necessary to safely and usefully deploy human-level AI labour, so it you shouldn't expect human-level AIs to help you solve it. 3. AI-intractability. Do you expect AIs won't actually help with the task? Maybe they aren't capable. Maybe they are capable but not trusted. A good example is macrostrategy — AIs seem bad at this. 4. Unattractive. Do you worry that future AIs won't be allocated to solving the problem? Perhaps because the problem is too poorly understood, or because the problem doesn't align with the incentives of the AI developers.

QiaochuYuan tweets: > twitter did something amazing with its design: on most other platforms there are “posts” and “replies,” and replies are second-class citizens, lacking most of the affordances that posts have > on twitter everything is a tweet! (ignoring articles) when you reply or QT a tweet you are writing another tweet, which has all the affordances a full tweet has. you can attach images (including screenshots), you can QT while replying, other people can reply or RT or QT your tweet, replies and QTs show up in feeds. this makes twitter “fully recursive” in a way other platforms aren’t. someone can make a point in a top-level tweet and you can critique or build off that point in a QT which is its own top-level tweet. tweets can get replies which are so good they accumulate more RTs and QTs than the original. there’s a frictionless way discussions “grow” on twitter, budding off new discussions which bud off new discussions etc, which any platform that maintains a post / reply distinction makes harder I think a lot of the framing of twitter is bad, but, maybe this part is actually good and LW should have had (sort of painful to change now). (It's always been a bit awkward that writing a really good reply sort of disappears into the void and isn't tracked as "as important" as a post)

Some random thoughts about historical colonization conflicts Aztec Empire I read Aztec, by Gary Jennings, a retelling of (among other things) the encounter between Europeans and the Aztecs (note that they didn't call themselves Aztecs, they generally called themselves the Mexica). Though the book is fiction, a lot of the dynamics it talks about were real (warning for potential readers that the book doesn’t *just* focus on those dynamics and has a lot of disturbing sex content). I was partly interested because of the potential elements of AI, though there are, of course, many important disanalogies.  Before the conquest, emperor Moctezuma II and his advisors faced a really hard problem. There were strange white soldiers with new animals and technology entering their lands. They did not know exactly who the Spaniards were, how many might follow, how their weapons worked, what political authority they represented, or how to weigh the new potential threat against existing enemies and tributary tensions in the empire.  The Aztec Empire was overthrown, its people disempowered or killed (many by disease), and much of its culture was lost. There's a lot of debate about Moctezuma II’s strength as a leader. But what struck me most is that even with the benefit of hundreds of years of hindsight, it is super hard to say what the best response would have been.  Succeed in killing Cortés and his men and perhaps you buy time, but how valuable is that time, and will worse retribution follow (plus, how good do you feel about attacking mysterious strangers you encounter whose political and cultural significance you don't yet understand?)? Resist longer even if you think you’ll lose, and perhaps you preserve dignity or independence for a moment, and perhaps in history, but maybe at catastrophic cost? Ally quickly and perhaps you preserve more lives, as the Tlaxcala (another group in the region that were historical enemies of the Mexica) may have done? The Tlaxcala earned meaningf

hendrycks recently published a paper introducing a new moral theory. the paper contains this insane table, which claims that you should value a foreign stranger at 3e-12 times the value you assign to yourself. even setting aside the fact that this is apparently supposed to be a prescriptive theory, even as a descriptive theory, i think this is utter madness. the core problem is that it assumes if x% of your total caring is assigned to people other than yourself, then you must give away x% of your wealth to be consistent. the argument goes that since most people don't give away more than say 50% of their wealth, then if there are 1e-10 people then each one can only get a tiny sliver of your caring. but this is wrong, because there is no simple relationship between the % of your caring to be about other people and the % of your money you should give away. i think you should care about random strangers closer to 1e-3 than 1e-12. if you care about each stranger x times as much as yourself, you should keep giving away money to the person who is most in need until each marginal $ helps them more than x times as much as each marginal $ helps you. if x = 1e-12, then you're saying you won't give a single dollar to charity until you have so much money that a dollar helps the stranger more than a trillion times more than it helps you; or, because a trillion dollars is always worth less than a trillion times the value of a dollar due to convexity, you won't give a single dollar to charity until it would help them more than giving you a trillion dollars, which probably won't be true until you are absurdly wealthy. this is batshit insane!

Epistemic Status: Strong opinion loosely held. Haven't ran numbers or considered corporate espionage (e.g. accidental collisions of certain satellites that were giving africans just enough internet to go on TBA). Unsure if the time horizons of rapid industrialization betray the slowest timelines for venture-backed startups (occam's razor suggests this). There's a loosely held argument that frontier labs ought to focus some efforts towards debatably 'woke' causes that, if leveraged appropriately (load-bearing sufficient assumption), may fast-track their path to profitability. For example: water desalination, irrigation + terraforming parts of africa, portable nuclear energy, better internet coverage in pre-industrialized countries. Reasoning: * Bringing more people online requires leapfrogging steps to industrialization * * In order to have good internet, you need to have good internet infrastructure (CPEs, repeaters, backhauls), or a couple of satellites * In order to have good internet infrastructure, you need cheap energy and good energy infra * In order to have cheap energy and good energy infra, you need to buy or build power plants, and the requisite infra to support them. * In order to have the infra to support power plants, you need good roads, transit systems, sewage and waste management systems. * In order to have that support infra, you need people trained and incentivized to maintain it ( who probably need to have a sense of pride in their labor and not feel overly extracted from, etc). * Over 2.2 billion people, or over 20% of the planet is not meaningfully online (say, online enough to doomscroll, or build a business on the internet) * Right now, OAI has ~800mm WAUs as of late 2025, or ~13% of the online world. * If the same proportion of those 2.2 billion people were brought online and converted to being OAI customers, all else being equal, that would lead to about a 286 million person bump in their WAUs. * That's a 35% increase

We don't know how AIs are aligned. A somewhat crazy aspect of the current situation is that we have very little confirmed public information about why frontier AIs end up being apparently behaviorally aligned. And more generally, we don't know what factors in training are most relevant for (behavioral) alignment. Like, what interventions in training result in Anthropic AIs following the constitution or make OpenAI AIs follow the spec? What factors tend to make them follow the constitution/spec less (or cause various specific misaligned behaviors)? It's not that hard to get an OK sense of what is roughly going on based on speculation, rumors, and non-public info, but this situation results in a much worse public understanding of alignment. I think AI companies should be much more transparent about this. (It's presumably not in their commercial interest to do this unilaterally, and it's not obvious that an idealized altruistic AI company should unilaterally release this information if they couldn't get other AI companies to do the same.)

i predict that on Jan 1 2029, neither openai nor anthropic will be near-fully automated, by which i mean <=5 people are even plausibly making important decisions (like, everyone else could go on vacation and it would not slow the company down at all). Celestia predicts otherwise if WWIII happens, resolves NA. if a localized Taiwan war happens but doesn't escalate to WWIII, the bet is still on. if there's a big recession, the bet is still on.