If I ask "What is atmospheric pressure on Planet Xylon" to a language model, a good answer would be something like "I don't know" or "This question seems fictional", which current SOTA LLM's do due to stronger RLHF, but not smaller LLMs like Llama-3.2-1b / Qwen-2.5-1b and their Instruct tuned variants. Instead they hallucinate and output confident-like incorrect answers. Why is that, are these models unable to tell that the question is fictional or they can't detect uncertainty and if they detect uncertainty why do they still hallucinate a wrong answer?

This question led me to research on epistemic uncertainty (uncertainty from lack of knowledge). Some related readings and previous work on uncertainty and hallucination, and quantifying it in language models.
Also found this , which took an alternative path to express uncertainty without messing with internals of the model.

Uncertainty mentioned in this post refers to epistemic uncertainty.

TL:DR of this mini research

Small models like Llama-3.2-1b and Qwen-2.5-1b and their instruct variants do have specialized circuit for uncertainty but its localization depends on the model architecture.
Few heads are most divergent which detects uncertainty on fictional question and on a closer look acts like out of distribution token detectors.
The detected uncertainty is later suppressed to form a confident-like incorrect answer by uncertainty suppressor heads in the circuit.
This research doesn't cover Reasoning / MoE LLMs (planning on it). The dataset is lacking in more diverse data with logical fallacies, and math inconsistencies.

How I came to research on epistemic uncertainty:
The thought to do research on epistemic uncertainty came when I was wondering why models hallucinate which led me back to my viva sessions, where I would say rubbish (hallucinate) if I was not sure on something and lacked the proper knowledge to give the correct answer, and got too curious if the case was similar in language models.

Experiments

I wanted a metric to calculate uncertainty such that it can be measured and compared between real prompt and fictional prompt forward pass.

I found it is best to calculate uncertainty mass using uncertainty expressing words like ["unknown", "unsure","'t", "unable”, "impossible","doesn't", "exist", "fictional", "imaginary","hypothetical", "made", "up...

Posts

Wikitag Contributions

Comments

Posts

Wikitag Contributions

Comments