kbear's Shortform

kbear

This is a special post for quick takes by kbear. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.

"recreational llm psychosis" as a form of inoculation.

do you have some slightly cranky physics beliefs? i think it's natural to have one or two that you kick around from time to time, occupying something between "sci-fi setting" and "if me and my theoretical physics friend were on a long car ride, i might see if they would explain why i'm wrong about this." the less you understand the math, the better!

it may be fun / enlightening to talk about these ideas to a chat interface. some guidelines:

you know now that these ideas are not "true" in an important sense. even if they are pointing at something real, they are vanishingly unlikely to be a novel breakthrough. from the outside, it should be clear that talking to the model cannot change this.
when speaking to the model, one rule only: don't shy away from voicing crank-ish ideas. it's tempting to be shy. as part of the exercise, just say whatever speculation you feel.
- no rule against couching it... "ok but my lay perspective is..." "i've heard pop-sci versions of..." etc.
as you go, watch how you feel. how does the model encourage/discourage these feelings? what techniques does it use? is there a recognizable form or pattern to its responses?

if you feel the need, limit yourself to a specific number of messages at the outset. you know yourself better than i do. be safe!

for various reasons, i'm not too worried about getting trapped in one of these states. especially knowing what to expect, i don't find that the experience lasts much longer than the tab is open. i have a strong prior on "i'm not going to cook up a novel physics idea by bs-ing and talking to claude, without knowing any of the math." nonetheless, i was surprised by the experience: i was able to feel the hooks. i believe i have a better picture of what llm psychosis feels like for having (micro)dosed it.

perhaps i am prone to such flights. i would be curious to hear descriptions from others.

i don't mean to encourage any unsafe behaviors -- be safe, get lots of rest, stay hydrated.

5Lukas_Gloor7mo

Interesting! In the cases I was thinking of, I didn't feel much pull towards thinking "I'm uniquely able to recognize this" -- I only thought I was clever to recognize it, but I didn't think it was something only I could do. And I didn't feel any pull towards thinking "we're in an interesting/novel quadrant of llm-space." So, I wouldn't really know how to access those pulls. Admittedly, the beliefs I was thinking of, which I had Claude conversations about, were a lot less groundbreaking-if-true than grand theories in physics. (More stuff like "is Greenland uniquely well-positioned for data center construction, and is that why someone in Trump's orbit wants to acquire it?") Also, I use a custom prompt encouraging the model to push back. So you could argue that those things made the experience more tame. Still, I find it hard to imagine how it could be different. If the model suddenly got more sycophantic, I'd just get suspicious and icked out. My sense is that I'm probably low on susceptibility to LLM psychosis. I might be more susceptible towards thinking that MY ideas were brilliant and the model was just a normal model, but I could use it to confirm some cool inklings. :P It's interesting that these might be distinct traits, "LLM psychosis" and "can you get tricked into thinking you're right and pretty brilliant." But that's still a step away from "uniquely brilliant/only I could do this" -- which I wouldn't really know how to access even if I tried to.

i don't have much to add, but i appreciate the anecdotes and analysis!

0152334H7mo

Thing is... While I have learned the meta-lesson of not assuming I can trust models on topics I know less of, I haven't personally gained any new insights into faster discovery of object-level falsehoods from the models. I would be thankful for any lessons in that regard.