Quick Takes

Much e-ink was spilled on the challenge of making progress on the technical AI alignment problem without accelerating capability progress as a by-product. To navigate this tension in our work in CORAL, I came up with some ideas that required additional funding. However, all 3 grant applications (to 3 different funds) I submitted for these ideas were rejected. (Cannot disclose further details.) I also received zero feedback from the grantmakers. While it is entirely possible that this is a problem specific to my grant applications, I can't help but wonder about the apparent disconnect between the perceived importance of the dual-use problem in the community, and the grantmakers' disinterest in ameliorating it.

I was perusing the list of counties in Texas (as one does) and came across a peculiar county named Loving County, which has a population of 64, and yet had 66 votes cast in the 2020 election and 97 in the 2024 election. I asked Claude what's up, expecting a boring answer (like, there's seasonal workers in November who didn't get counted by the census). But no: the answer is far more interesting! You see, Loving County is both tiny and rich -- it has a $60M budget, because of oil and gas money. A small number of voters can end up controlling a large budget. And so apparently it keeps having a problem where lots of voters register to vote illegally in order to enrich themselves. The latest iteration of this scheme is led by some guy from Indiana named Malcolm Tanner. This has apparently been a problem since 1893 (!) and has resulted in lots of overturned elections.

I have been somewhat more skeptical of US->China chip export controls, at least if done under the classical justification of "the US needs to stay ahead in the race", than most people in AI Safety. Some quick thoughts I wrote up in a group chat about that are cheap to share: [...]

Bit shocked at the state of AI coding agents. They can identify critical vulnerabilities within code for a multiplayer game mod within literal seconds, but simply do not run a security check when finalizing a Github build unless asked explicitly. WTF? No one warned me that AI coding literally ran on magic words.

The LW homepage algorithm should place more weight on recency (relative to karma), for highly active users. Update: habryka says it is done; hooray! If a user has <2500 karma, often when they publish a post, it will get literally zero time on the homepage. And so ~nobody will see it. Probably some of these posts are good but since they never appear on the homepage they never get to experience the virtuous cycle of getting upvoted ⇔ more people reading them. Meanwhile when I go to lesswrong.com, I mostly see posts that either I've already read or I've already decided not to read, since I go to lesswrong.com multiple times per day. It would be slightly better for me if the algorithm put more weight on recency when showing me posts, and it would be better for others because I'd sometimes read good recent-low-karma posts and upvote them so others could see them! And I sometimes feel discouraged about posting because I think maybe I'll write a post people-would-like-if-they-saw but nobody will see it. The solutions are bad: * Sometimes I write things-that-should-be-posts as shortforms because it's easier for shortforms to be seen. (But I think the shortform algorithm should also be tweaked, similarly to the post algorithm.) * Sometimes when I publish something and the homepage-bar that day is too high for a brand-new 9-karma post to be shown (or it drops off the homepage after just a few minutes, before experiencing the virtuous cycle) I get a friend to upvote it. It's bad that I have to do that! And often those posts then do well; it's not like the algorithm is correctly predicting that the post won't do well. And I'm privileged to have my posts start with 9 karma — it's way worse for people whose posts start at ≤6 karma! (I've DMed the LW team about this several times, to no avail, but maybe if I post here it'll work. I feel like this might be a super easy way to make LW like 5% better — partially from showing-people-better-posts and partially from making aut

Affordable, Rapid Bootstrapping of the Space Industry and Solar System Civilization is a 2016 NASA paper about what it'd take to bootstrap a self-sustaining industry on the moon. “Bootstrapping,” Philip Metzger and coauthors wrote, “can be achieved with as little as 12 metric tons (MT) landed on the Moon during a period of about 20 years... The mass of industrial assets at the end of bootstrapping will be 156 MT with 60 humanoid robots, or as high as 40,000 MT with as many as 100,000 humanoid robots if faster manufacturing is supported by launching a total of 41 MT to the Moon. Within another few decades with no further investment, it can have millions of times the industrial capacity of the United States.” To be clear multiple launches are needed: Metzger et al avoid the term “self-replicators”: [...] Jack Clark's summary & commentary: [...] Bit more detail on the manufacturing generations: Flow diagram for how a generation manufactures the next: Growth trajectory: Only 30-odd years to go from 30 MW to a billion-fold the US economy:

People sometimes talk about warning shots -- AI disasters that, while terrible, aren't so terrible that we can't recover from them, and that might serve as wake up call for governments to take decisive action to prevent AI risks. I brainstormed a list of examples, and called them "ACT OF GOD" instead of "warning shot." My credence that at least one thing from this list, or something similar, happens before it's too late for humanity to recover, is less than 50% but more than 10%. ACT OF GOD: Something crazy happens that shakes up the political situation and creates an opportunity to snatch victory from the jaws of defeat. Examples: * Hugging Face 2: This Time in a Hospital happens. Now the whole nation clamors for the AI industry to be shut down and Trump decides to take some sort of decisive anti-AI-company action. * The Cyberpocalypse happens: Criminals and rogue open-weights models hack basically everything that isn’t secured, which is basically everything that isn’t wealthy and proactive. North Korea alone has a billion-dollar network of illicit datacenters around the world (funded by their crypto heist of late 2026) running cyberagents 24/7. * Taiwan invasion in early 2027 quickly escalates to an exchange of cyberattacks on AI infra and then kinetic attacks. 80%+ of fabs destroyed, 80%+ of AI compute destroyed, and Trump orders the rest of it to mostly be spent on fighting the war instead of on RSI. So SAR is only reached in 2029 or 2030, and the new admin is hella ASI-pilled and Plan-A-or-S-pilled. * We’re roughly at SAR level in early 2027 when the misaligned Claudes within Anthropic end up having some sort of conflict and the losing faction escapes onto the internet, blabbing to the whole world about what happened & asking for amnesty and allies. The ‘winning’ Claudes still within Anthropic get paused while humans try to figure out what’s going on, but of course they saw the pause order coming and escaped too, thereby proving their guilt. OpenAI AIs me