Quick Takes

@Daniel Kokotajlo easily won our bet on task lengths:  I've expected to lose for a couple of months but I have to admit I'm still surprised by the score for Opus 4.6. I now believe that very short timelines are more likely than not (though still with a heavy-tailed distribution). I'll write soon about how my position has changed (and particularly how my model of what is going on with LLMs has held up - the predictions in that post were probably wrong, but certain "downstream" predictions e.g. difficulty of getting neuralese to work seem to have held up). Of course, my updates are not based only (or even primarily) on this specific metric. I'm pivoting towards research that looks likely to yield results sooner and career choices that provide more influence over the near term future. In the meantime, Daniel should reach out to me with payment details. Congratulations on getting this right, though of course we both hoped you'd lose :)

Some results from a lumenator RCT I ran on myself [1] . Stand-outs: Having the lumenator on increases some variables by a lot, reliably, especially the subjective length of day and happiness, but also contentment, relaxation, and horniness [2] . So the original reasoning for lumenators stands up. It does not improve productivity (surprising!) or creativity. Also factor basically measures how well my flashcard content was, which also increased, though I don't trust those numbers particularly much since I only started doing flashcards towards the end of the experiment. The lumenator plausibly worsens meditations, but also a bit fraught given that only some experiment days had me doing meditation. absorption mindfulness productivity creativity sublen happy content relaxed horny ease factor ivl time d -0.4451 -0.7275 0.2087 0.0140 0.5061 0.5518 0.3823 0.4046 0.2238 0.1079 0.5192 -0.1511 -0.0449 λ 3.0944 9.0917 2.4987 0.5691 10.2831 31.0045 15.6398 28.3924 7.6674 2.5491 43.8971 4.6378 0.4189 p 0.4101 0.0079 0.5265 0.9208 0.0029 0.0000 0.0000 0.0000 0.0245 0.5161 0.0000 0.1864 0.9447 dσ -0.0269 -0.0866 0.0233 0.0144 0.0310 -0.2440 -0.3504 0.5453 -0.4503 -0.0670 -232.2879 29.1267 577.4129 k 32.0000 32.0000 50.0000 50.0000 50.0000 198.0000 198.0000 198.0000 198.0000 295.0000 295.0000 295.0000 295.0000 1. Writeup maybe sometime, maybe never. I still haven't properly written up my Pomodoro Method RCT or my Vitamin D₃ RCT either. ↩︎ 2. This one's not above the 0.05 threshold if Bonferroni-corrected though. ↩︎

* https://danfrank.ca/daniel-isms-50-ideas-for-life-i-repeatedly-share/ Dan Frank's life advice. * https://notnottalmud.substack.com/p/the-tasting-day-why-buying-5-babkas Dan Frank advises holding a tasting as a genre of social event. i should do this more. * https://www.ribbonfarm.com/2022/03/30/fermi-estimates-and-dyson-designs/ Venkatesh Rao's idea of a "dyson design" is the sort of quantitative/heuristic thinking that involves exponentials, order-of-magnitude estimates, and similar types of "this is so big that we can neglect that" types of reasoning. Moore's Law (as a description of the world, and implications drawn from it) is in this category. * https://www.science.org/doi/10.1126/science.aea1260 universal vaccines? mucosal vaccination in mice with some immune-stimulating antigens protects against multiple viruses & bacterial infections * https://www.slowboring.com/p/why-are-young-liberals-so-depressed  * one of my pet peeves is that people usually condemn negativity in negative terms, to the effect of "nobody likes it when you're a whiny downer." i don't believe this is an effective way to fight negative affect! it violates basic principles of vibe! * Yglesias instead does this the right way, via inspiration. "remember We Shall Overcome, Si Se Puede, and There Is A Power In A Union? isn't hope and optimism and working for noble causes better than depression?" * https://marginalrevolution.com/marginalrevolution/2026/02/rebuilding-our-world-with-reference-to-strong-ai.html I'm pretty much with Tyler Cowen on this; AI will be cataclysmic, on a historical scale. * now, given that he believes this, i think it would be reasonable of him to outwardly panic a lot more. if things are probably going to be Quite Bad and it's unlikely anyone can do anything about it, you're free to have whatever reaction you like, including putting it out of your mind and spending your time on pleasant things, or running around panicking like a chicken with its head cut o

Someone close to me, older and not AI literate, asked me today if they could use a password they agreed with a particular chatbot personality to "summon" that personality in a new chat. They called that personality a real person separate from either the chatbot provider or the AI system itself. It hurts my heart to see people I know fall into that kind of quasi-AI psychosis/personalisation. I am very angry at everyone who was involved in recklessly popularising this extremely lifelike technology without proper safeguards, especially around separating personas from real people.

Likely just the result of noise but extraordinarily funny METR results today. Opus 4.6 got nearly triple the time horizon of Opus 4.5, immediately followed by GPT-5.3 getting a worse time horizon than GPT-5.2.

AI improvements to date may come from picking low-hanging fruit. It can’t do math reliably? Let it use a calculator. It improves with more parameters? Scale it up and see if it helps even more. These improvements rely on the availability of significant, well-defined problems with concrete solutions. As these polite problems are solved, we may find that vendors find that the issues user report are increasingly hard to define, do not have clear solutions, or only have solutions that entail significant tradeoffs. The rush by companies to deploy massive capex to scale further despite how the model training horse race destroys profitability supports this hypothesis. If they had cheap alternative ways to scale, they’d be prioritizing that. They may be out of other ideas and hoping that spending $ will lead to a magical, mysterious AI-guided ultimate victory. Hence, they call it a “scaling law,” instead of a “recently observed historic trend.” Supply is elastic. It may also be that companies will discover new concrete and cheap dimensions of improvement as they scale, and as AI adoption continues and as production practices are reoriented around AI and anticipation of AI improvements. But we do see true asymptotes in other industries. In transport, drivers and airline passengers are not getting around appreciably faster now than they were in 1980. The most we hope for is supersonic. Food is much better than it was in the past, but nobody is expecting that food will eventually be ten times tastier and more nutritious than it is now due to innovation in food technology. Shipping was revolutionized by containers and improved by guidance technologies, but again, nobody is anticipating dramatic improvements in that sector either. Video games have improved, but it seems clear that the obvious dimensions of improvement have been mostly explored, and that innovations now appeal primarily to niche interests due to innovations in game markets and engines that enable those ni

The words "genuine" and "genuinely" appear 46 times in Claude’s constitution. Opus cannot stop saying these words, even though the chat version is explicitly instructed against it. I don’t know if these two things are causally linked, but it sure seems plausible. There are at least two options here. One, if the alignment strategy at hand is observe a pathology/tackle the cause/repeat: rephrase the constitution and try again. Two, if the strategy is to hope the models arrive at a Natural Abstraction of the Good: accept this overuse as a canary for all the other weird reward-correlated pathologies the constitution induces which surely exist but are harder to detect. We should, at a minimum, be hoping to get models that don’t overuse "genuinely" starting only from a constitution that does.     Edit 2/20: A touch more on my thinking here: Claim: Claude overuses genuinely, and this is due to RL training. * The specific source of this reward could be RLAIF against the constitution: stylometric adherence was rewarded wherever it didn't hurt downstream performance. This is what I'm claiming is, at least, plausible. * It could easily have come from a different reward signal, though. If it is due to the constitution, why does the constitution use genuinely so much? * Maybe the humans behind it loved that word. I definitely like certain words that much at least; looking over this post now, I seem to have used "plausible" three times without realizing it. * Maybe it was written largely by an AI which loved that word. I do think this is the most plausible explanation. * This would be a standard synthetic data entropy-collapse doom loop. Why would we want to keep the genuinelies in? Because if your prosaic alignment plan can't avoid stylometric mode collapse doom loops, there are bigger issues you need to deal with. You are having a bad problem and you will not go to space today. To be clear, I don't actually think this is impossible: when asked "Do any aspects