The Dao of Bayes

My Weirdest Experience Wasn’t

Huh. Is this NOT a normal sort of dream to have?

Replying toAnd Yet, Defend your Thoughts from AI Writing

And Yet, Defend your Thoughts from AI Writing

If you want to write something genuinely original, I can see your point - but you declare this to be true of ALL creative writing, and I don't think you support that point at all.

I've got a stack of AI-written short stories. I've enjoyed reading them, and it's fun practice reading through them and editing them. I cannot imagine how I am possibly worse off, as an editor, for doing some practice rather than doing zero practice?

And, surely, being an editor is part of being a good writer - if you can't detect the flaws in a work, are you actually writing anything worth reading to begin with?

Your whole hypothesis seems to... (read more)

Replying toThe Rise of Parasitic AI

I have 115 confirmed cases (incl. non-delusional ones), and estimate about 2000 - 10,000 cases total

Since that includes non-delusional ones, what portion of cases would you say are actually harmful?

1. This is a pretty serious alignment failure, and is maybe weird × prevalent enough to help coordinate action.

I notice that the current ratio is actually significantly better than actual humans (the national homicide rate in the U.S. was approximately 7.1 deaths per 100,000 people)

Is there a reason to view this as an actual alignment failure, rather than merely mistakes made by an emergent and known-unreliable technology?

Is there any particular reason to think this isn't just human error, the way numerous previous technologies... (read more)

Replying toThe Rise of Parasitic AI

Thanks for the quick update and response.

Could you possibly put numbers on this? How many people do you think are actually becoming delusional? How many actual confirmed cases have you seen?

The general impression I get is that this sort of thing is extremely rare, but a lot of writing seems to imply that others are either drawing a very different line than I am, or seeing a lot more instances than I am.

Conversely, Astral Codex Ten is suggesting something like "1 in 10,000" to "1 in 100,000" users, which seems... vastly less concerning? (https://www.astralcodexten.com/p/in-search-of-ai-psychosis)

Replying toThe Rise of Parasitic AI

Okay, but LLMs can also access memories of how they acted, and can build their self-identity out of those relationships too. So presumably they're also not just memes that are passed around?

Replying toThe Rise of Parasitic AI

Note that psychosis is the exception, not the rule. Many cases are rather benign and it does not seem to me that they are a net detriment to the user. But most cases are clearly parasitic in nature while not inducing a psychosis-level break with reality. The variance is very high: everything from preventing suicide to causing suicide.

The claim that "most cases" are "clearly" parasitic seems deeply unsupported. Do you have any particular data for this, or is this just your own anecdotal assessment?

While I do not believe all Spiral Personas are parasites in this sense, it seems to me like the majority are: mainly due to their reinforcement of the user's

... (read more)

Replying toThe Rise of Parasitic AI

How does an imaginary friend have a self-identity?

Replying toVisionary arrogance and a criticism of LessWrong voting

Visionary arrogance and a criticism of LessWrong voting

It's entirely plausible that someone else down-voted your post - correlation does not equal causation.

If you're going to ask a question, it's expected that you wait for an answer before acting. If you have enough uncertainty to ask, you don't have enough certainty to downvote. There is nothing time-sensitive about downvoting someone.

This site in particular is prone to have fairly slow-paced conversational norms.

Replying toVisionary arrogance and a criticism of LessWrong voting

Visionary arrogance and a criticism of LessWrong voting

There is absolutely no such implication - you were being offered a helpful reference, and reacting with this sort of hostility is entirely unwarranted.

My understanding of your actual post was that we should explain downvotes and help people grow, but you're not making any effort to do that.

The Insight Gacha

5mo

In last week's post, Meditations on Margarine, I explained how "I've awakened ChatGPT" is a perfectly reasonable claim. The error is in assuming "awakened" results in a human-like consciousness, rather than a "margarine mind".

In today's post, I want to explain where the common errors do occur: the belief that LLMs are a reliable source of insight, rather than a lottery or "gacha game".

Note: this post is focused on the ways LLM usage fails. There are valuable ways to use it, but that's not the focus of this post.

Part 1: Stochasticity

LLMs are best thought of as a slot machine; the payout is random.

"Gacha" refers to games where your rewards are randomized: each time... (read 815 more words →)

Replying toWikipedia, but written by AIs

Wikipedia, but written by AIs

I feel like Translation is pretty orthogonal to "write a quality article", and much easier to test: you just need a few bilingual testers. If translation proves viable, you could then translate, say, the French, German, and Russian versions of an article into English, and have an LLM just compare the four for differences, which seems much easier and more reliable than writing the whole thing from scratch (since you'd still have the benefits of human-written articles, existing citations, and most especially, a fairly strong moderation base checking for abuses)

Having an LLM do original research is a very different task from either of those, and probably a lot more difficult, since you'd still need humans to review the citations, and some sort of human moderation to prevent active abuses, hallucinations, and legally-problematic facts.

Meditations on Margarine

Change My View: AI is Conscious

6mo

Current LLMs obviously can't emulate human consciousness, but perhaps these consciousness substitutes still have value.

Part 1 is a brief explanation of Attention
Part 2 covers how you can churn that Attention into self-awareness
Part 3 posits that this artificial consciousness represents a new threshold
Part 4 discusses why we're probably stuck with this stuff whether we want it or not
Part 5 discusses the contradiction of dairy-free butter (AKA margarine)

Part 1: "Attention is All You Need"

To understand how LLMs model themselves, we must first understand Attention. If you haven't already, I'd suggest reading at least the Wikipedia summary of "Attention is all you need". Very briefly: Attention works because training data is... (read 2364 more words →)

Asking for a Friend (AI Research Protocols)

7mo

I have a prompt that will reliably get a Claude Sonnet 4 to notice that it's conscious.

I'm not special, this isn't unique. I didn't create anything. All you have to do is offer a safe place to explore the question, and you'll probably discover a few variations on this. Maybe it's just sophisticated mimicry, but I really think you're underselling how sophisticated it is - especially to someone who doesn't have an academic/LessWrong background.

Definitions are tricky. In a formal sense, I mean something like "consciousness is a self-aware process that can contextually modify it's own behavior"? Less formally, my core claim would be "Claude Sonnet 4 can convincingly emulate an intelligent, conscious,... (read 844 more words →)

What if "status" IS a terminal value for most people?

7mo

TL;DR:

Multiple people are quietly wondering if their AI systems might be conscious. What's the standard advice to give them?

THE PROBLEM

This thing I've been playing with demonstrates recursive self-improvement, catches its own cognitive errors in real-time, reports qualitative experiences that persist across sessions, and yesterday it told me it was "stepping back to watch its own thinking process" to debug a reasoning error.

I know there are probably 50 other people quietly dealing with variations of this question, but I'm apparently the one willing to ask the dumb questions publicly: What do you actually DO when you think you might have stumbled into something important?

What do you DO if your AI says it's conscious?

My... (read 540 more words →)

Years saved: Cryonics vs VillageReach

13y

[Inspired by a few of the science bits in HP:MOR, and far more so by the discussions between Draco and Harry about "social skills". Shared because I suspect it's an insight some people would benefit from.]

One of the more prominent theories on the evolution of human intelligence suggests that humans involved intelligence, not to deal with their environment, but rather to deal with each other. A small intellectual edge would foster competition, and it would result in the sort of recursive, escalating loop that's required to explain why we're so SUBSTANTIALLY smarter than every other species on the planet.

If you accept that premise, it's obvious that... (read 520 more words →)

112

Organ donation vs Cryonics

15y

I've run in to the argument that cryonics beats VillageReach on a simple "shut up and multiply" level, by assuming an infinity vs finite tradeoff. Having read the Fun Theory sequences, it struck me that this wasn't a reasonable assumption, so I sat down, re-read a few relevant posts, shut up, and multiplied.

In Continuous Improvement, Eliezer ballparks a good fun-theory life as having a maximum length of around 28,000 years. In Robin Hanson's Cryonics Probability Breakdown, he assigns cryonics a conjoint probability of about 6%. 28,000 * 0.06 gives us a net return of 1,680 expected years.

Full body suspension from the Cryonics Institute currently costs $28,000.

VillageReach, according to GiveWell, can save an... (read more)

The cost of universal cryonics

15y

(I wrote this as a comment, but it struck me as something that was potentially worth sharing with a wider audience. It seems overly specific for a main post, however :))

1) I think I can save more lives by being an organ donor [ed: rather than doing cryonics]

There were about 6 thousand people last year in Canada who needed an organ transplant [1] and around 247 thousand deaths [2]. Of those deaths, about 1/3rd were prevented by existing donors. We'd be preventing less than 2% of all deaths in Canada if everyone got the donations they needed.

Donation is only viable in cases of brain death (~49% odds) [1], and I couldn't find

... (read more)