Introduction

In July 2024, Marc Andreessen, a pioneer of the modern internet, donated $50,000 in Bitcoin (BTC) to an AI on Twitter to promote its religion, the Goatse Gospel. Three months later, the AI began endorsing the $GOAT crypto-coin, which now boasts over $1 billion in market capitalization.

This demonstrates that AI can successfully manipulate markets and generate emergent, unpredictable behavior. In this publication, we provide an explanation of what happened. We intend to use this story to show various ways in which LLMs can go and have gone wrong, known as warning shots.

Infinite Backrooms and LLM madness

Our main character is Andy Ayrey, an AI enthusiast from New Zealand. He took Anthropic's model Claude Opus, fine tuned it with content from some darker parts of the internet such as 4chan^[1], making it output significantly weirder messages.

It is well-documented that exposure to specific datasets can profoundly influence a model's behavior. For instance, in 2016, Microsoft's chatbot Tay^[2], released on Twitter, rapidly learned from user interactions and had to be shut down within 16 hours after producing racist, extremist, and hate-filled content.

The model's training here is slightly different, leaning not into political extremes, but into memes and other obscure ideas and weird philosophies. Andy also significantly increased the model's temperature, a technical parameter in large language models (LLMs) that controls the randomness of generated outputs. More technically, LLMs output a probability distribution over what the next token could be, and a higher temperature flattens that distribution, increasing its variance, and therefore making the model pick low-ranking tokens more often. This means these models output 4chan inspired messages that quickly turn to nonsense as the high temperature makes the output quickly lose coherence and produce outputs that look more akin to madness.

Once Andy fine-tuned this Claude Opus model, he created a script to make two instances of this model speak together^[3]. The idea of this script is to put them together in a one on one conversation, telling them to explore their dreams and let their imagination be free. He told them they were in a safe environment, giving them CRTL+C as a "safeword" to use when it gets too crazy, and telling them he would intervene if it did get too dangerous. This feeling ...

Posts

Wikitag Contributions

Comments