Quick Takes

Claude Code sometimes hallucinates user messages. Over the last couple of days, @Puria and I have noticed that Claude Code will sometimes hallucinate messages from users. So far, we’ve observed this happening when CC is operating autonomously in a loop to monitor training runs. Unprompted, at some point between checks, CC sends a message to itself prepended with “Human:”, and then acts on that message. These messages sometimes tell CC to change its monitoring behaviour, and CC will make the requested change. Example 1: Example 2: Curious if other people have noticed similar behaviours, especially outside of autonomous monitoring loops.

My wife and I bought one of the Nectome pre-sale discount cards. This is the second five figure purchase I have made based pretty much entirely on a highly-upvoted LessWrong post. I've also lost 20 lbs in the last year and have more energy than ever (despite having a newborn at home) by injecting myself with research chemicals that I heard about on LW, and have taken investment advice from people here that has paid for everything above and more. Our polygenically-screened kid is 3 months old. I am ambivalent about the term superbabies in general, but he is definitely "super" to us already, whatever the effect of applying a little extra selection pressure on his genes turns out to be. The jury is still out on Nectome, though I think my fellow Max H (a different person!) makes a pretty good case for them, even (or perhaps especially) if you have short AI timelines. More generally, while I am uncertain about playing to your outs as a strategy for averting AI doom (or just what it actually looks like concretely), I think a similar sort of idea works quite well as a strategy for living a good life in the face of it. Now that I have a dependent, I am signing up for life insurance and following other standard advice, as well as doing the things that I think should be standard practice for responsible parents, despite the fact that, on my own inside views, a lot of this stuff is pretty negative EV (more so than the normal way that insurance is always negative financial EV for the buyer). I also derive enormous benefit from the original Sequences, and from actively participating in the discourse here these days: seeing interesting takes, sharpening my thoughts and ability to express them in writing, emotional stability, introspective ability, (well-calibrated) self-confidence, etc. All this is to say, a heartfelt thank you to everyone here, especially the authors and commenters of all the posts linked above, as well as @habryka, the Lightcone team, and Eliezer for maki

I recently realized that self-improvement wouldn't work without dopamine detoxing (at least for me). I've tried all sorts of ADHD medication and self-improvement techniques like journaling, GTD, 5 secs rule, mindfulness, counseling, etc., none of them stuck. I aim to be more self-disciplined by instilling them into my life, but they all require self-discipline to enforce. Especially if I'm surrounded by superstimuli like social media and meme videos, anything that requires attention away from them becomes extra annoying and painful. When my brain is adapted to superstimuli, it also takes more willpower to shift my attention to something less stimulating or addictive. So I procrastinate more. I wait until the last moment when intense stress can override my craving for dopamine spikes. This also means I gradually lost my agency. I had to wait for external crises to push myself out of indulging in the superstimuli. But self-improvement and contributing to meaningful causes require serious thinking and work, beyond the bare minimum effort to avoid failing at whatever I'm doing. Only recently, after I went cold turkey on all the superstimuli, did I finally feel free again. Doing serious work is no longer that excruciating. I reclaimed mental space for more reflection. The flywheel of self-improvement finally starts spinning.

i haven't had a chance to think deeply about it but vibes wise i don't like activation oracles

Is there some simple way to mute Inkhaven posts or do I need vibecode something? (I find LW notably less useful during Inkhaven)

Repentance seems to be very rare among the powerful. I tried to search with multiple LLMs and in other ways for examples where a king or a dictator realized the evilness of some of their past actions, realized their rule is not justified, and voluntarily resigned. I have not found a single example of this happening. There are some examples of kings and dictators voluntarily resigning, but it's usually motivated by being tired of ruling (often for health reasons), and very occasionally genuine support for democracy. But as far as I can tell, it's never because a ruler realized the evil of their ways. I also searched for crime bosses, warlords and successful large-scale fraudsters who voluntarily gave up their evil ways due to repentance. Again, there were hardly any examples. People sometimes repent in prison; people sometimes turn themselves in to the police when they see they will soon get caught anyway; and people sometimes retire from crime to a safer life-style, keeping their ill-gotten gains to themselves. I only found two examples of successful criminals changing their ways while still successful due to a change of heart - Nicky Cruz who was a gang leader in New York, and General Butt Naked, a Liberian warlord. And even there, I'm a bit suspicious - many of General Butt Naked's stories of his previous horrific atrocities seem false, and I wonder what else is false in his story. I'm interested if people can find better examples of evil leaders and successful criminals repenting while still in power, I would be relieved to see more examples of this happening. I find the rarity of repentance of the powerful a very sad fact about human nature, and it makes me less optimistic that current dictators and unscrupulous politicians will significantly change for the better if given superintelligent AI advisors. Of course, one can still be an okay or even maybe a good ruler without ever repenting their evil actions in the past, but I still don't feel great about thi

A historical note re: MIRI vs EA When I was arguing against MIRI trying to build Friendly AI (now "aligned ASI") circa 2012, I think 3 people at MIRI were thinking the most about strategy and occasionally responded to me: @lukeprog @CarlShulman @Eliezer Yudkowsky. Only one of them actually changed their mind about this (namely Eliezer), while the other two left MIRI and don't support AI pause to this day, AFAIK. (Luke now "leads our grantmaking on AI governance and policy" at OpenPhil/Coefficient. Carl became an advisor to OpenPhil and is now Research Director at Situational Awareness LP.) Seem significant (although I'm not sure of the implications) that the "try to build safe ASI, don't bother trying to coordinate a pause" strategy originated at MIRI and only a minority of people there who defended it actually changed their mind. (It's also curious how quiet both Luke and Carl are these days, at least in public.) EDIT: To clarify, Carl does seem to support some kinds of short pauses under some future circumstances. (Quote: "To the extent you have a willingness to do a pause, it’s going to be much more impactful later on. And even worse, it’s possible that a pause, especially a voluntary pause, then is disproportionately giving up the opportunity to do pauses at that later stage when things are more important.") (@Connor Leahy I thought of this while reading your You can only build safe ASI if ASI is globally banned, which reminded me of my past arguments to MIRI.)