Quick Takes

Bertrand Russell's parents died by the time he was four years old and in 1876 he went to live with his grandfather, the last Whig prime minister, who as a young man had met Napoleon at Elba. In 1966 at the age of 94 he met Paul McCartney and converted him to his anti-war stance on Vietnam. Previously, I had figured that this lifespan (roughly 1870 to 1960) was the most extreme length of history for someone to live through. You have to be born early enough to remember a time when big cities didn’t have street lights or automobiles, while ideally living to see the atom bomb and first man in space (1961). For example: * Churchill (1874–1965) rode in Britain’s last great cavalry charge (Omdurman 1898) and later in life ordered Britain’s first hydrogen-bomb program. * W.E.B. DuBois (1868–1963) was born three years after the Civil War but died before the March on Washington. Other such lifespans were Laura Ingalls Wilder (1867–1957), Frank Lloyd Wright (1867–1959), W. Somerset Maugham (1874–1965), Picasso (1881–1973). On Twitter people pointed outside the West, the same lifespan was even crazier, if they were born into totally pre-industrial worlds e.g. * “Syngman Rhee (1875-1965) who was one of the last people in Korea to pass the old Confucian civil service examinations and later became the President of South Korea, playing a pivotal role in the early Cold War … He was born in a preindustrial world where the old Confucian social order still held sway and nobody around him had ever seen a train and died in a world of jet planes and manned space travel.” @avrilbradley23 * “A curious Surmic born then would have seen the forging of the Ethiopian Empire, the end of the witch-chiefs, Christian missionaries, the Italian conquest, literacy, new crops, and maybe the moon landing.” @Peter_Nimitz Meanwhile I had figured I was going to live through the “Great Stagnation” and the world would gradually become a humongous nursing home. But it seems like AGI is likely to ke

Anthropic releases new post about RSI and slowdowns-- also commits to doing verification research and verification-focused policymaker engagement. (In my view, this is probably the best post/announcement about AI risks that has been produced by any frontier AI company in recent memory). https://www.anthropic.com/institute/recursive-self-improvement [...]

Campaign update: With 56% of the vote in, AI safety champion Scott Wiener and Connie Chan (both Democrats) have advanced to the November election. Wiener got 41.3% of the vote, compared to Chan's 28.6%. This is roughly what I expected. Wiener is favored to win in November, but it's not a done deal, for a few reasons: * Voters for the third-place finisher, Saikat Chakrabarti, might go disproportionately to Chan over Wiener: I think Chakrabarti is more ideologically similar to Chan overall, though it's not totally clear. * Republicans really don't like Wiener, and might prefer Chan over Wiener. * Nancy Pelosi, who endorsed Chan, will probably use what power she has to try to further boost Chan. Wiener is about 70% to win per prediction markets, and that sounds about right to me. (Maybe it's a tad low; my intuition is more like 75%.)

ControlAI has launched an official campaign in Canada, with 33 politicians across party lines backing a clear statement in favor of an international prohibition against developing superintelligent AI: https://controlai.com/canada-statement/en I used to be ambivalent about government intervention wrt AI safety, because politics can screw things up. However, it seems clearly positive to me now: the development of superintelligence impacts the whole human species, and as such, the democratic process should have a say in whether (and how) superintelligence is developed. Governments have been tolerant thus far, but the general public is not so happy about this, meaning that political campaigns have a good chance of success (in my estimation). Without such government intervention, the AI industry is on-track to develop superintelligence within the century if not the decade, and I do not believe safety research is ready to handle the risks. As Yudkowsky put it, only law can prevent extinction. Public reception of this campaign will make a big difference in the willingness of further politicians to endorse this or similar statements. Although I would normally hesitate to put signal-boost-style politics on LW, this could use a signal boost, and is quite relevant to the audience here.

Over the last month or two I have rarely found it worth finishing any posts on the front page (even when I have done so, I usually felt ambivalent about the choice in retrospect). I am not sure whether I am simply busier with other things or whether the quality of the front page has went down, but as an incredibly avid lesswrong reader, it seems worth it for me to flag this bare fact (without proposing any particular explanation).

How do LLMs and humans compare with regards to the amount of energy, data, and compute that's used to train/run them? I was inspired by Samuel Knoche's post on sample efficiency to come up with some numbers. This table was made by Opus 4.8. after some iteration: "Task" = "produce one thoughtful ~1,000-token answer"; unclear if this is a useful number, obliviously it doesn't generalize. I do think it's interested to compare the energy/computer ratio for inference between human and AI. Training data for humans is a big "???". There's ~4x10^8 waking hours. Opus 4.8 cited this 2024 paper which says "our sensory systems gather data at ~1⁢0^9 bits/s". That gives ~4x10^17 as an upper bound. But human sensory data is extremely redundant. If someone wants to spend time figuring out a good way to estimate entropy adjusted information here, that would be cool. (FWIW, I spent a little over an hour on this -- I did a fair amount of iteration with GPT5.5, looked a bit myself into a few numbers, and got new instances to do estimates from scratch to see if things lined up.)

Getting elected involves compromising your values. Usually, when people say "compromising values", it carries a connotation of low integrity. That's not really my intention here. Instead, I mean it in a more neutral way: if you're running for office, it's pretty likely that your values will be out of step with your constituents' values in one way or another. Maybe you have a wider moral circle of concern, or have a stronger sense of justice, or whatever, leading to you holding views that are unpopular among your constituents. This makes you less likely to win. And so, you have three choices: 1. Stick to your values, tanking the decreased likelihood of winning 2. Change your values, or at least commit to acting on your constituents' values rather than your own 3. Deceptive alignment: Run on your constituents' values, but once you're in power, act on your own values Options 1 and 2 trade off against each other, and different politicians have reputations for being at different points on the spectrum. For example, Bernie Sanders is thought of as pretty close to 1, while Gavin Newsom is thought of as pretty close to 2. The most clean-cut example of #3 that I know of is Jimmy Carter, who deliberately tried to appeal to segregationists in his campaign for governor, but then famously declared that "the time for racial discrimination is over" in his inaugural address. I find this interesting to think about, because I think that racial discrimination is bad, but at the same time I think it's a bad policy to be deceptively aligned to your constituents. Anyway, I sometimes think about this in the context of electing AI safety champions to Congress. I think the value proposition is great, but the more they take route 2 over route 1, the lower the value of electing them. (Meanwhile, I think that AI safety champions should steer away from route 3, though in my opinion it's fine to place a different emphasis when campaigning than when policymaking.)