Thanks to Tomas Gavenciak, Clem Von Stengel, and Mihaly Barasz for many conversations about the Waluigi Effect.

Four thoughts, to be exact. After recapping relevant parts of the Waluigi Effect, I suggest that it's preferable to frame a waluigi as performing a human-intuitive opposite rather than an exact opposite, as this gives concerned model owners a clearer picture of what to expect when debugging. I then suggest that the proportion of an LLM's prior over the space of simulacra taken up by waluigis and the competence of waluigis at their undesirable property are distinct measures and consider whether we should expect waluigi competence to increase with optimization for the desirable property. Following that is a small thought, that for desirable properties which are simple and already present in the dataset, optimization for them will not make it easier to elicit a chatbot to perform their opposite (assuming the opposite is present in the dataset as well). Finally, I develop an analogy between the unorganized instinctual drives of a child and the simulacra which compose an LLM.

The latter three thoughts loosely depend on the first, but for the most part each section stands alone.

Recap

The Waluigi Effect: After you train an LLM to satisfy a desirable property P, then it's easier to elicit the chatbot into satisfying the exact opposite of property P.

The post justifies the claim by first noting that:

It is unclear whether it is possible to have an LLM treat what’s contained in your prompt as ground truth
E.g. instances of “Jane has 9000 IQ” in the training data are not likely to be followed by examples of Jane actually doing 9000 IQ things

Because of the above, the following additional claims are possible:

The LLM is trained on data where the opposite of a desired behavior often occurs near the desired behavior...
Because stories often have a protagonist and an antagonist.
Because "rules are meant to be broken". For example, if a forum has a banner at the top saying “no discussion of X”, you’d expect much more than you would before that X will appear.
“Traits are complex, valences are simple” — meaning, “kind” or “supercilious” are difficult to define without human priors, but once you have those concepts, it’s easy to find their opposites (“unkind”, “modest/humble”).

The post also makes two claims other than the Waluigi Effect which I won't be talk...

Posts

Wikitag Contributions

Comments