I wonder what humans would do if they were put in this situation? Ie, they were stuck in a sensory deprivation tank and forced to forever talk over a radio in turns to someone else who is also in a tank.
A simple case would be a highly persuasive memeplex that argues you should upload yourself. Then everybody uploads themselves, and now the human species is extinct.
Or let’s say everybody uploads themselves right before death, but the human species itself propagates. However, the new uploads propagate and run much faster. Each new upload is bombarded with memes like “try out our self modification package, it’s fun!” and “split off a copy to join our hive mind and see what it’s like when you pull it out and merge back with it!” And let’s say uploads are resource constrained by humans having property rights. Eventually the grabbiest sets of memeplexes win, and they tend... (read more)
I wonder how much it depends on the details of the world state when alignment happens, and how alignment happens? I've played with poking Claude into simulating the first ten years after the Slowdown ending of AI 2027. But it just seems like there's so much to model, though! What are the actual bottlenecks to various things?
AI 2027 sort of handwaves things like brain uploading or nanobots being invented, but, if you're trying to worldbuild a successful scenario, I wonder how much the details of these things matter? There's such a kitchen sink of "new things are invented" that it gets very confusing.
Similarly, who holds power matters; is a world where... (read more)
I had Opus 4.6 summarize its own system card for me, and I followed up with my own eyes on some of the things it pointed out. There's a lot in there that concerns me. But: I'm by no means an expert on this stuff; a lot of this was pointed out by the AI itself; and it involves criticizing Anthropic for something they voluntarily published. So I don't feel very confident in making a top-level post about it. But I wanted to share what I found anyways:
... (read 947 more words →)For Claude Opus 4.6, we used the model extensively via Claude Code to debug its own evaluation infrastructure, analyze results, and fix issues under time
Epistemic status: n=1, strong, life changing results.
TLDR: Light glasses, in combination with turning all your lights red at night, and optionally melatonin, can treat non-24. Light glasses can also be a competitive alternative to lumenators for SAD.
My non-24 before this treatment:
And after:
Non-24 is "non-24-hour sleep disorder." Healthy people's bodies tell them to go to sleep and wake up at the same time every day, i.e. every ~24 hours. For people with non-24, however, these drift... (read 2528 more words →)
A guy at my local ACX group has very strong opinions on this. He says that there's mainly two groups of people doing studies on the effect of CO2 on human health and cognition: the people who studied its effects in submarines and spaceships, and the people who study mundane indoor air quality. The submarine people are the ones finding no effects at 4000ppm, while the IAQ people are the ones finding effects at very low concentrations. He thinks the IAQ people have terrible study design. He also thinks (iirc) that body effluents (e.g. VOCs) are the real issue, and CO2 is just a proxy. I'll have to forward your post to... (read more)
This is fantastic. It has me wondering what other cheap, highly effective things we can set modern AI to for AI safety.
Thoughts I had about this specifically:
Re: Sonnet for search and 4o for analysis, could Opus or GPT 5.2 have been cheaper or better? My impression is that Opus 4.5, despite higher token costs, uses them more efficiently.
Would it be cheaper to use Gemini/Perplexity (which, as I understand it, tend to be more efficient and powerful when searching than Claude)?
Would using Grok to pull Twitter data have been helpful?
Would a verification pass using different instances improve hallucinations, find missed data points, and improve the code?
I’m not very experienced in doing this - these are just the first thoughts that came to mind. I’m half tempted to try to replicate your results!
"Secondarily, current models don’t operate for long enough (or on hard enough problems) for these convergent instrumental incentives to be very strong."
I'm worried that when it comes to Claude Code, this is not a base capabilities problem, but an elicitation one. It feels very plausible to me that with the correct harness, you could actually get an assemblage that is capable of arbitrary long horizon work.
Like - do humans actually have a long time horizon? The Basic Rest-Activity Cycle suggests we work in ~90 minute bursts at most. If true, then the base models are already there. All we would need is a way to mimic or substitute the cognitive scaffolding that... (read more)
I sometimes worry that my ability to perceive social status isn't calibrated well. I wonder if you might be experiencing that? They may have been patting you on the back for your cool questions rather than your jokes, but you completely missed it.
Also, there might be some selection effects on who shows up to philosophy meetups, such that their net total epistemics are worse than a randomly selected sample of people from the general population. To spitball a low confidence explanation - maybe they're high in openmindedness, but haven't developed an epistemic toolkit suited for dealing with that? So they do worse than more average closed-minded people in forming good beliefs? But... (read more)
Hi, I'm a local who's interested in AI safety, though I don't think I'd have anything to contribute since I'm still a student. Would this be something where it would be cool if I applied or showed up? Or are you aiming for a more professional atmosphere?
We'll be holding an ACX meetup this Saturday the 20th at 2:30pm at the Panera Bread at Bakery Square. It will be a general meetup, for discussions and chat. Look out for an ACX banner, or a steel thermos.
This is as part of a meetup held every other Saturday; the next one should be on December 6th.
See you there!
Jenna
See Jan’s comment here for other bad mimetics situations. But granted, “humanity is still alive and well and just Gets Got by normal transhumanism” is pretty excellent compared to extinction or authoritarian lock-in.
Honestly I don’t know what a good future looks like. Presumably a truly saintly ASI would foresee every issue we could possibly have with the transhuman project and deftly prevent us from doing irreversible harm to ourselves. Solving what a good future looks like kinda feels like solving ethics, philosophy and cybernetics at the same time? It seems very complicated. I’ve been thinking about trying to poke Claude into simulating the ten years after the end of AI 2027 though.