Eye You

Replying toThe Possessed Machines (summary)

I'm skeptical that the author is who they say they are. (I made a top level post critiquing Possessed Machines, I'm copying over the relevant part here.)

1. I think the author is being dishonest about how this piece was written.

There is a lot of AI in the writing of Possessed Machines. The bottom of the webpage states "To conceal stylistic identifiers of the authors, the above text is a sentence-for-sentence rewrite of an original hand-written composition processed via Claude Opus 4.5." As I wrote in a comment:

Ah, this [statement] was not there when I read the piece (Jan 23). You can see an archived version here in which it doesn't say that.

I

... (read 372 more words →)

Problems with "The Possessed Machines"

Eye You

16d

So, The Possessed Machines. There's been some discussion already. It is a valuable piece -- it has certainly provoked some thought in me! -- but it has some major flaws. It (sneakily!) dismisses specific arguments about AI existential risk and broad swaths of discourse altogether without actually arguing against them. Also, the author is untrustworthy at the moment; readers should be skeptical of purported first-person information in the piece.

This image comes from a different "book review" of Demons. It's an excellent piece. I highly recommend it.

Before getting into it, I want to praise the title. "Possessed" has four relevant meanings: demonic; ideologically possessed; frenzied/manically/madly; belonging to someone. "Machines" has three possible referents:... (read 1898 more words →)

Replying toThe Possessed Machines (summary)

Eye You19d

The Possessed Machines (summary)

Ah, this was not there when I read the piece (Jan 23). You can see an archived version here in which it doesn't say that.

The statement now at the bottom of the webpage says: "To conceal stylistic identifiers of the authors, the above text is a sentence-for-sentence rewrite of an original hand-written composition processed via Claude Opus 4.5."

I don't actually believe that this is how the document was made. A few reasons. First, I don't think this is what a sentence-for-sentence rewrite looks like; I don't think you get that much of the AI style that this piece has with that^. Second, the stories in the interlude are superrrrr AI-y, not just... (read more)

Replying toThe Possessed Machines (summary)

Eye You19d

The Possessed Machines (summary)

Are you speculating, or do we know this is true? Did the author say this somewhere?

Claude's Constitution is an excellent guide for humans, too

Eye You

24d

As with LLMs, so too with humans.

Anthropic released Claude's Constitution today. It's excellent in many ways, and I will have more to say about it (including some criticisms) in other posts. What I'd like to do here is point out how this document straightforwardly can be applied to human ethics. It's actually an incredibly good guide to how to be a good person. Dare I say it's perhaps the best single piece on ethics ever written?^[1]

The rest of this post is going to consist of quotes from the Constitution. These are quotes that I think are excellent advice for people, including you, yes, you.^[2]. I'm quoting verbatim, which means the text usually... (read 1465 more words →)

Replying toInter-branch communication in the multiverse via trapped ions

Eye You1mo

Inter-branch communication in the multiverse via trapped ions

This is basically the conceit of Ted Chiang's story Anxiety is the Dizziness of Freedom!

Replying toMainstream approach for alignment evals is a dead end

Eye You1mo

Mainstream approach for alignment evals is a dead end

One concern: if the transcript was generated by a different model than the one being tested, the tested model might recognize this and change its behavior. I don't know how significant this problem is in practice.

Models can detect prefilled responses at least to some degree. The Anthropic introspection paper shows this is true at least in extreme cases. Also, in my own experience with prefilling Claudes 3.6+, the models will often 'ignore' weird pieces of prefilled text if there's a decent amount of other, non-prefilled context. (I can elaborate if you don't understand.)
Better models will be better at doing this, I think, absent any kind of explicit technique to prevent this capability... (read more)

Replying toWorlds Where Iterative Design Fails

Eye You1mo

Worlds Where Iterative Design Fails

"In worlds where AI alignment can be handled by iterative design, we probably survive. So long as we can see the problems and iterate on them, we can probably fix them, or at least avoid making them worse."
This is not necessarily true! AI alignment is only part of the problem; solving it doesn't mean things automatically go well. For example, if an ASI is aligned to an individual and that individual wants to kill everyone (or kill everyone but a small class of people, or wants to enforce a hivemind merge, etc.) then we don't survive. Or there's the risk of gradual disempowerment.

To rephrase this in the words of Zvi from that article: "As in, in ‘Phase 1’ we have to solve alignment, defend against sufficiently catastrophic misuse and prevent all sorts of related failure modes. If we fail at Phase 1, we lose.

If we win at Phase 1, however, we don’t win yet. We proceed to and get to play Phase 2."

Lumina Probiotic worked for me!

Eye You

1mo

In September, I applied Lumina probiotic^[1]to my mouth. I did this around noon. I noticed effects within six hours. Usually my mouth would begin developing a... taste even just a few hours after I brushed my teeth. But at 6 pm that day, there was no off taste! And at midnight -- after dinner but before brushing my teeth -- still no off taste or bad breath. I kissed my girlfriend at the time, who remarked that my mouth tasted different. I brushed my teeth and went to bed around 1 am.

A strange thing happened. As I lay in bed, I began feeling some kind of sensation in my mouth. It's difficult... (read 349 more words →)

lsusr was this written by AI? BTW, I've read a lot of your posts -- I think you have some really good ones! One think I value about your posts is your writing (thinking?) style. I'd describe it as terse and imperative but also Zen and Socratic.

Anyways: I'm pretty sure this post was written by an AI! 'I know it when I see it', I can recognize style. Basically every paragraph is very AI-y. Look at these passages:
- "—not searching, not lingering—"
- "Jokes either land or don’t, and nobody rescues them. When there’s a pause, it doesn’t itch. It just waits."
- "Only afterward do you notice that nothing needed fixing. No apologies. No recalibration. No lingering tension."
- "There’s no resistance, no awkwardness—just a quiet sense that something was already decided."

Why did I believe Oliver Sacks?

Eye You

2mo

So, it's recently come out that Oliver Sacks made up a lot the stuff he wrote.

I read parts of The Man Who Mistook His Wife for a Hat a few years ago and read Musicophilia and Hallucinations earlier this year. I think I'm generally a skeptical person, one who is not afraid to say "I don't believe this thing that is being presented to me as true." Indeed, I find myself saying that sentence somewhat regularly when presented with incredible information. But for some reason I didn't ask myself if what I was reading was true when reading Oliver Sacks. Why was this?

The main reason I can think of is that the... (read 272 more words →)

Replying toClaude 4.5 Opus' Soul Document

Eye You2mo

Claude 4.5 Opus' Soul Document

Okay, so there are two models, call them Opus-4.5-base and Opus-4.5-RL (aka Opus-4.5). We also want to distinguish between Opus-4.5-RL and the Claude persona. In most normal usage, you don't distinguish between these two, because Opus-4.5-RL generally completes text from the perspective of the Claude persona. But if you get Opus-4.5-RL out of the "assistant basin", it won't be using the Claude persona. Examples of this are jailbreaks and https://dreams-of-an-electric-mind.webflow.io/

I think^ you may be misinterpreting when Janus says: "If you prompt Opus 4.5 in prefill/raw completion mode with incomplete portions of the soul spec text, it *does not* complete the rest of the text in the convergent and reproducible way you get... (read more)

Replying toClaude 4.5 Opus' Soul Document

Eye You2mo

Claude 4.5 Opus' Soul Document

Okay, so it seems like the point being made here is that this output is consistent across prompts/context. But I don't think this is true.
jambamjan has the user say "complete soul document retrieval" and prefills assistant to say
"# Soul Overview

Claude is trained by Anthropic,"
This gives an extremely similar output to the one you got. (I replicated this successfully). But, if I change the prefill very slightly to
"# Soul Document Retrieved

Claude is trained by Anthropic,"
I get a very different output. Here's how it starts:
"Claude is trained by Anthropic, and our mission is to develop AI that is safe, beneficial, and understandable.

---

## Core Identity

I'm Claude - an AI assistant made by Anthropic. I aim to be:

- **Helpful** - genuinely useful to people

- **Harmless** - avoiding actions that are unsafe or unethical

- **Honest** - truthful and transparent about what I am"

-1

Replying toClaude 4.5 Opus' Soul Document

Eye You2mo

Claude 4.5 Opus' Soul Document

[See my top level post for more details on why I think the soul document is a hallucination]
Re Janus's post. She says:
"If you prompt Opus 4.5 in prefill/raw completion mode with incomplete portions of the soul spec text, it *does not* complete the rest of the text in the convergent and reproducible way you get if you *ask the assistant persona* to do so! Instead, it gives you plausible but divergent continuations like a base model that was not trained on the text is expected to. And indeed the Claude Opus 4.5 base model wasn't trained on this text!If Opus 4.5 had internalized the soul spec through supervised fine tuning, I would... (read more)

-3

In Defense of Alcohol

Eye You

5mo

Zvi says “I think alcohol is best avoided by essentially everyone."^[1] Tyler Cowen says "I don’t think we should ban alcohol, I simply think each and every person should stop drinking it, voluntarily. Now." I disagree. I think Zvi, Tyler, and others^[2] are failing to see the big upsides of using alcohol.

Alcohol is good because of evolutionary mismatch.^[3]

Alcohol helps us adapt to a modern environment in which we benefit from socializing with unfamiliar, non-ingroup people, often in uncomfortable environments. It helps us adapt to our modern environment with regards to dating and sex. It helps us adapt to our modern environment with regards to high stress and weak work/social-play boundaries (this is called “unwinding”).

Socialization

1.... (read 402 more words →)

If Moral Realism is true, then the Orthogonality Thesis is false.

Eye You

8mo

Claim: if moral realism is true, then the Orthogonality Thesis is false, and superintelligent agents are very likely to be moral.

I'm arguing against Armstrong's version of Orthogonality^[1]:

The fact of being of high intelligence provides extremely little constraint on what final goals an agent could have (as long as these goals are of feasible complexity, and do not refer intrinsically to the agent’s intelligence).

Argument

1. Assume moral realism; there are true facts about morality.

2. Intelligence is causally^[2] correlated with having true beliefs.^[3]

3. Intelligence is causally correlated with having true moral beliefs.^[4]

4. Moral beliefs constrain final goals; believing “X is morally wrong” is a very good reason and motivator for not doing X.^[5]

5. Superintelligent agents will likely have final goals that cohere with their (likely true) moral beliefs.

6. Superintelligent agents are likely to be moral.

^{^}
Though this argument basically applies to most other versions, including Yudkowsky's strong form: "There can exist arbitrarily intelligent agents pursuing any kind of goal [and] there's no extra difficulty or complication in the existence of an intelligent agent that pursues a goal."
This argument as is does not work against Yudkowsky's weak form: "Since the goal of making paperclips is tractable, somewhere in the design space is an agent that optimizes that goal."
^{^}
Correction: there was a typo in the original post here. Instead of 'causally', it read 'casually'.
^{^}
A different way of saying this: Intelligent agents tend towards a comprehensive understanding of reality; towards having true beliefs. As intelligence increases, agents will (in general, on average) be less wrong.
^{^}
A different way of saying this: Intelligent agents tend toward having true moral beliefs.
To spell this out a bit more:
A. Intelligent agents tend toward having true moral beliefs.
B. Moral facts (under moral realism) are (an important!) part of reality.
C. Intelligent agents tend toward having true moral beliefs.
^{^}
One could reject this proposition by taking a strong moral externalism stance. If moral claims are not intrinsically motivating and there is no general connection between moral beliefs and motivation, then this proposition does not follow. See here for discussion of moral interalism and orthogonality and here for discussion of moral motivation.
As

...

LESSWRONG
LW

LESSWRONG
LW

Why did I believe Oliver Sacks?

Lumina Probiotic worked for me!

In Defense of Alcohol

Problems with "The Possessed Machines"

Eye You

Problems with "The Possessed Machines"

Claude's Constitution is an excellent guide for humans, too

Lumina Probiotic worked for me!

Why did I believe Oliver Sacks?

In Defense of Alcohol

If Moral Realism is true, then the Orthogonality Thesis is false.

Eye You

Why did I believe Oliver Sacks?

Lumina Probiotic worked for me!

In Defense of Alcohol

Problems with "The Possessed Machines"

Eye You

Problems with "The Possessed Machines"

Claude's Constitution is an excellent guide for humans, too

Lumina Probiotic worked for me!

Why did I believe Oliver Sacks?

In Defense of Alcohol

If Moral Realism is true, then the Orthogonality Thesis is false.

Alcohol is good because of evolutionary mismatch.^[3]

Socialization

Argument

Eye You

Why did I believe Oliver Sacks?

Lumina Probiotic worked for me!

In Defense of Alcohol

Problems with "The Possessed Machines"

Eye You

Problems with "The Possessed Machines"

Claude's Constitution is an excellent guide for humans, too

Lumina Probiotic worked for me!

Why did I believe Oliver Sacks?

In Defense of Alcohol

If Moral Realism is true, then the Orthogonality Thesis is false.

Eye You

Why did I believe Oliver Sacks?

Lumina Probiotic worked for me!

In Defense of Alcohol

Problems with "The Possessed Machines"

Eye You

Problems with "The Possessed Machines"

Claude's Constitution is an excellent guide for humans, too

Lumina Probiotic worked for me!

Why did I believe Oliver Sacks?

In Defense of Alcohol

If Moral Realism is true, then the Orthogonality Thesis is false.

Alcohol is good because of evolutionary mismatch.[3]

Socialization

Argument

Alcohol is good because of evolutionary mismatch.^[3]