Josh Levy

Abstract Whereas previous work has focused primarily on demonstrating a putative lie detector’s sensitivity/generalizability[1][2], it is equally important to evaluate its specificity. With this in mind, I evaluated a lie detector trained with a state-of-the-art, white box technique - probing an LLM’s activations during production of facts/lies - and found...

Introduction Whereas hallucinating is unintentional, lying is intentional. To actively lie, you need to be able to: 1) recite facts 2) know what makes facts factual and 3) be able to modify them accordingly. Until recently, open source LLMs have not been capable of lying reliably. Models like OPT and...