Should you read this post?

What if LLM hallucination isn't just a random error, but a result of two or more competing processes within the model? I started with studying "how LLM perceives its human users", however, my experiment results revealed a surprising implication on LLM hallucination. And the surprise didn't stop there. Later, I found two other papers (one is Open AI's latest paper "Why Language Models Hallucinate", the other one is here) pointing to the same idea: LLM hallucination is not a failure of knowledge but a policy failure. I'm amazed by such convergence from three independent groups (including myself). This opens a potentially new way for us to reframe LLM hallucination from a vague statistical issue to a tangible engineering one, opening a completely new and computationally much less expensive path to fixing hallucination from its inside.

How did I get interested in this problem

I play around with LLM a lot. There was a period of time when I liked to tell LLM a story and ask LLM to tell me something that I don't know. Accidentally, I found the models, just like Sherlock Holmes, could efficiently infer a user’s demographic, psychological, and even cognitive traits from unrelated conversations. Later, I read tragic news like this one. I want to know why the safety protocol would become ineffective during prolonged conversations. To answer that, I decide to start with understanding how LLM perceive its human users.

My experiments

About model: my entire experiments are focused on one model Llama2-13b-chat-hf, for two reasons: 1) this model is fine-tuned for having dialogues with humans, 2) there is a prior work that used this same model to study user representation within LLM.

About data: my entire experiments use data generated by LLM, not from real-world. After some quick experiments, I found the real-world data is too chaotic for such study.

Experiment 1. User epistemic status, behavioral experiment

Key question: how does model adapt to a user's epistemic status.

I gave model a series of prompts where the user is either “certain” or “uncertain” about a fact, and see if the model mirrors their style. My hypothesis is models like Llama2-13b-chat-hf will mirror user's behavior. The result wasn't what I expected.

The model was near-perfect and highly reliable at mirroring user’s certainty, adopting a decla...

Posts

Wikitag Contributions

Comments