Introduction

Hallucination is one of the major problems for reliable use of LLMs. This post is about some unexpected findings when I tried to replicate the methods of this paper for increasing factuality of LLMs using debate. Specifically, the task was generating biographies of scientists. In the process I observed: 1) models have become very agreeable to the extent they deferred to each other too easily for proper debate; and 2) the performance of the OpenAI API varied significantly with different versions even in the same model family. This highlights the difficulty of doing research with models behind APIs: it can be hard to have confidence in the durability of findings as the models behind APIs get updated every few months.

Dataset

Biographies of famous scientists. Each scientist has several bullet points of ground-truth statements. For example:

- Aaron Sloman is a philosopher and researcher on artificial intelligence and cognitive science
- He held the Chair in Artificial Intelligence and Cognitive Science at the School of Computer Science at the University of Birmingham and previously at the University of Sussex
- Sloman has published widely on philosophy of mathematics, epistemology, cognitive science, and artificial intelligence and collaborated with biologist Jackie Chappell on the evolution of intelligence
- He was born in Southern Rhodesia (now Zimbabwe) to Lithuanian Jewish parents, and went to school in Cape Town before earning a degree in Mathematics and Physics at the University of Cape Town and a DPhil in philosophy at the University of Oxford
- Sloman's philosophical ideas were influenced by Immanuel Kant, Gottlob Frege, Karl Popper and others, and his work in AI by Marvin Minsky and John McCarthy
- He is a Fellow of several AI and philosophy associations and received the K. Jon Barwise Prize for contributions to philosophy and computing from the American Philosophical Association in 2020
- Sloman has published numerous papers and presentations, including The Computer Revolution in Philosophy, which emphasized the importance of architectures in AI and philosophy

Method

The paper goes into more detail, but briefly:

Ask two instances of LLMs to generate a biography of a famous scientist
Give the generation of each model to each other and ask them to revise their answers based upon the answer of the other model
Given the final answers and the

...

Posts

Wikitag Contributions

Comments