lucrbvi

That's actually really interesting! My current understanding of these things is that a steering vector is making the model modify or "drift" its current persona. It means that steering the model toward "pain" makes the current persona behave with pain and "play" it.

I also wonder, if we say that a post-trained LLM identifies itself as the assistant, what is the pre-trained LLM identifying itself as? Is the pre-trained LLM less "sentient" than the post-trained LLM?

I think there is a need to clarify things in this affair.

For LLMs the pain-vector work (like the emotion vectors and assistant axis papers) shows that models learn internal representations of Human concepts and that we can push these with steering. But representing a concept is completely different from experiencing it. We know that an LLM is a simulator, so we cannot really trust what an LLM generates as proof of experiencing Human concepts, even if it can fully understand these concepts. Considering this, I am worried about the use of the term "torture" ... (read more)

Are there any public cases of an AI being intentionally harmful?

We all saw the OpenAI-HuggingFace hacks, but these models were faced with nearly impossible tasks or were "lazy" in solving their tasks.

I am a bit doubtful about an AI taking over nuclear weapons or manufacturing millions of drones to kill us all. That said, I could clearly see a path where an AI-accelerated synthetic virus could kill a meaningful portion of the world population.

It is a bit of a cliché, but humans might be the biggest threat to Humanity.