This work was performed by Rhys Gould, Elizabeth Ho, Will Harpur-Davies, Andy Zhou, and supervised by Arthur Conmy.
This has been crossposted from medium, but has been shortened here for brevity.

TLDR: we reverse engineer how GPT-2 small models temporal relations; how it completes tasks like “If today is Monday, tomorrow is” with “ Tuesday”. We find that i) just two heads can perform the majority of behaviour for this task ii) heads have consistently interpretable outputs iii) our findings generalise to other incrementation tasks involving months and numbers.

Our results lead to a circuit consisting of a single head capable of number incrementation and also performing a vital role in time tasks. Notation for such diagrams is defined in the Model Architecture section.

The Appendix contains a table of notation for reference and some resources for those new to mechanistic interpretability.

All experiments can be found in the Colab notebook.

The Task

We focus on interpreting how GPT-2 small can solve the task of next day prediction. More specifically, the task we focus on involves providing the model with prompts of the form:

Some remarks:

We refer to a particular prompt, such as <day> = “Monday”, as the Monday -> Tuesday prompt.
When we talk of the subject day for a prompt, this refers to the day that is present in the prompt (<day> as above), where the correct day is the answer to this prompt (the day after <day>).
When we talk of the correct prob of a prompt, such as Monday -> Tuesday, it refers to the probability of the correct answer, such as “ Tuesday”, given by the model.

Passing the 7 sentences into the model correctly solves the task, with correct probs:

Here there are 7 probabilities for each of the 7 tasks ordered respectively, i.e. the 0th entry is the correct probability for the “Monday -> Tuesday” prompt, the 1st entry is the correct probability for the “Tuesday -> Wednesday” prompt, etc.

We now give a quick overview of the decoder-only transformer model, which GPT-2 small is an instance of.

Model Architecture

Here we provide a brief description of the architecture of decoder-only transformers and develop some notations to be used throughout the post.

For notational ease we count from 1 here, but outside of this section, counting is done from 0.

At a high level, the model looks like:

Tensors are represented by rounded rectangles and disp...

Posts

Wikitag Contributions

Comments