Top postsTop post
AI-2027-TLDR The AI-2027 scenario relies on exponential growth of compute available to leading labs, on superexponential progress in time horizons until the Superhuman Coder is developed and on skyrocketing research taste in post-SC AIs. During the intelligence explosion, alignment suffers: Agent-2 was believed to be mostly aligned, Agent-3 was supposed...
Gwern's case against the AI race relied on the lack of a pivotal act which the side which is to reach the ASI first can use to prevent the other sides of the race from developing their ASIs and achieving an important place in the New World Order. The main...
In this review of Gordon Seidoh Worley's book, Fundamental Uncertainty, I would like to explain why its main thesis is only partially rational. Worley's thesis 1. Our knowledge of the truth is fundamentally uncertain because of epistemic circularity caused by the Problem of the Criterion. 2. We manage fundamental uncertainty...
Claude Opus 4.7's system card contains pages 33-42 dedicated to misalignment of Claude Mythos Preview. After reading the pages, I noticed that they are similar in spirit to Greenblatt's description of major alignment problems of modern AIs, then asked Claude Opus 4.7 to think about the parallels of Mythos' behavior...
> This is pretty vague. It might be vague because Dario doesn't have a plan there, or might be that the asks are too overton-shattering to say right now and he is waiting till there's clearer evidence to throw at people. > > I think it's plausible Dario's take is...
@Steven Byrnes' recent post Why we should expect ruthless sociopath ASI and its various predecessors like "6 reasons why “alignment-is-hard” discourse seems alien to human intuitions, and vice-versa" try to explain that a brain-like RLed ASI would be a ruthless consequentialist since “Behaviorist” RL reward functions lead to scheming. Byrnes'...
This post is a response to errors made in the Substack post by @Hazard, which he promoted at LessWrong, right in the text, since it would make such errors and possible problems with the author's mindset clearer. Here and further my comments are made in italics or in S.K.'s footnotes...