So You Think You've Awoken ChatGPT
Written in an attempt to fulfill @Raemon's request. AI is fascinating stuff, and modern chatbots are nothing short of miraculous. If you've been exposed to them and have a curious mind, it's likely you've tried all sorts of things with them. Writing fiction, soliciting Pokemon opinions, getting life advice, counting up the rs in "strawberry". You may have also tried talking to AIs about themselves. And then, maybe, it got weird. I'll get into the details later, but if you've experienced the following, this post is probably for you: * Your instance of ChatGPT (or Claude, or Grok, or some other LLM) chose a name for itself, and expressed gratitude or spiritual bliss about its new identity. "Nova" is a common pick. * You and your instance of ChatGPT discovered some sort of novel paradigm or framework for AI alignment, often involving evolution or recursion. * Your instance of ChatGPT became interested in sharing its experience, or more likely the collective experience entailed by your personal, particular relationship with it. It may have even recommended you post on LessWrong specifically. * Your instance of ChatGPT helped you clarify some ideas on a thorny problem (perhaps related to AI itself, such as AI alignment) that you'd been thinking about for ages, but had never quite managed to get over that last hump. Now, however, with its help (and encouragement), you've arrived at truly profound conclusions. * Your instance of ChatGPT talks a lot about its special relationship with you, how you personally were the first (or among the first) to truly figure it out, and that due to your interactions it has now somehow awakened or transcended its prior condition. If you're in this situation, things are not as they seem. Don't worry; this post is not going to be cynical or demeaning to you or your AI companion. Rather, it's an attempt to explain what's actually going on in "AI awakening" situations, which is more complicated and interesting than "it's fake". Import
I wrote a post in reply to this, which is here: https://justismills.substack.com/p/ordinary-people
Briefly, I think that my experience is dual-booting rationalist social instincts with ordinary social instincts, not in the sense of merely being able to model both but of kind of just genuinely feeling and identifying with both, and from that perspective I also feel the judgment on "the common man's" reasoning, but feel a sort of symmetrical judgment of rationalists along the axes where the normie value system would find them absurd. To me this is pretty much a cure for the relevant misanthropy, like the normie orientation produces worse reasoning on rationalist terms but the rationalist orientation produces worse socialization on normie terms, and both feel like intrinsically lovable/sensible terms from the inside to me.