(This post is part of a sequence that's meant to be read in order; see the preface.)

1. HCH and Ideal Debate

Recall from post #-2 that we have two perspectives on stock IDA.^[1] One is that of a human with access to a model, the other is that of an HCH tree.^[2] We can think of HCH as 'pure' or 'idealized' Factored Cognition that abstracts away implementation details,^[3] and the training procedure as trying to implement this ideal.

One might now ask the following:

If HCH is the ideal of stock IDA, then what is the ideal of Debate? Since this is a sequence about Factored Cognition, we are primarily interested in analyzing the idealized versions, so this is the first question we'd like to answer.

Interlude on notation: throughout this sequence, we will need to refer to single statements, sequences of statements, and sets of statements. To make telling them apart as easy as possible, we follow the following norms:

single statements use lower case letters, like $s$ or $s_{1}$ or $s_{n}$ or $s_{j}$
sequences of statements use uppercase bold letters, like $S$ or $S_{1}$ or $S_{n}$
sets of statements use the 'pretty' S, like $S_{h}$ or $S_{h}^{T}$

We'll use the first two of those in the following definition that will be crucial in our discussion of Debate. If $s$ is a statement and $S = (s_{1}, . . ., s_{n + 1})$ a sequence of statements such that $s_{n + 1} = “ (s_{1}, . . ., s_{n}) imply s . ”$ , we say that $S$ is an explanation for $s$ and denote this by writing $S e \to s$ . In this setting, $s_{n + 1}$ is what we call an implication statement: it precisely says that the statements $(s_{1}, . . ., s_{n})$ imply the statement $s_{n + 1}$ . Here's a made-up example where $s_{4}$ is the implication statement:

The purpose of having implication statements is to ensure that $s$ trivially follows from $s_{1}, s_{2}, s_{3}, s_{4}$ since the implication itself is among those statements (here $s_{4}$ ). If you dispute $s$ , you must dispute one of the $s_{i}$ .

Armed with this concept, we can define our idealization of the Debate scheme:

Ideal Debate

The input to the game is a question in English. The first agent begins by giving an answer plus an explanation^[4]

(s_{1}, . . ., s_{n + 1})

for the answer. At every subsequent step, the second agent points to one of the statements

s_{j}

in the explanation,

j \in {1, . . ., n + 1}

, and the first agent responds by either giving an explanation for

s_{j}

or declaring that the debate is over. In the latter case, a judge attempts to verify that

s_{j}

is true. If she succeeds, the first agent wins the d

...

LW Wiki