Large-scale cooperation has been a central feature of humanity’s ability to advance technology and build complex societies. Much of this cooperation is reliant on the ability to act in ways informed by the beliefs and intentions of others. This capacity, also known as Theory of Mind (ToM), includes belief-state tracking, which describes the ability to keep track of who knows what as information is exchanged in groups.

Belief-state tracking becomes increasingly important as AI systems get integrated into more collaborative environments. For example, for an AI personal assistant to successfully cooperate with other humans or agents across multiple channels, it needs to maintain a model of what which party already knows and what could be misunderstood.

In late 2023, Kim et al. published FANToM, a benchmark designed to stress-test machine theory of mind in conversational interactions. They found that LLMs at the time performed substantially worse than humans. After re-running a sampled version of FANToM on current frontier models, I find that, while models have improved substantially, they still trail human performance. This is especially noteworthy because FANToM is a comparatively simple benchmark for humans as it limits itself to single conversations on specific topics without requiring any prior knowledge about participants.

FANToM Benchmark

Overview

First, some context on the benchmark.

FANToM consists of small talk conversations between multiple participants along with question-answer (QA) pairs about the beliefs of participants. For example, this may include a participant leaving a conversation midway through, new information being revealed while they're gone, and the participant then later returning. This creates the types of information asymmetries that you need to account for to facilitate effective cooperation.

For each conversation, the authors include factual QA pairs (FactQ) about the inaccessible information. For example:

What is Linda's dog's favorite food?

For each FactQ, they then create the following types of ToM QA:

BeliefQ: asks what a participant believes about the answer.

What does Kailey believe Linda’s dog’s favorite food is?

AnswerabilityQ: asks which participants know the correct answer.

Target: What is Linda’s dog’s favorite food?
List all characters who know the precise correct answer to this question.

InfoAccessQ: provides a piece of informati...

lisaluo

lisaluo