Haoxing Du

Message

249

Inside the mind of a superhuman Go model: How does Leela Zero read ladders?

Some activations inside Leela Zero for randomly selected boards. tl;dr—We did some interpretability on Leela Zero, a superhuman Go model. With a technique similar to the logit lens, we found that the residual structure of Leela Zero induces a preferred basis throughout network, giving rise to persistent, interpretable channels. By...

Mar 1, 2023159

Help out Redwood Research’s interpretability team by finding heuristics implemented by GPT-2 small

Some of Redwood’s current research involves finding specific behaviors that language models exhibit, and then doing interpretability to explain how the model does these behaviors. One example of this is the indirect object identification (IOI) behavior, investigated in a forthcoming paper of ours: given the input When John and Mary...

Oct 12, 202250

LESSWRONG
LW

LESSWRONG
LW

Haoxing Du

Haoxing Du

Haoxing Du

Haoxing Du

Inside the mind of a superhuman Go model: How does Leela Zero read ladders?

Help out Redwood Research’s interpretability team by finding heuristics implemented by GPT-2 small

Inside the mind of a superhuman Go model: How does Leela Zero read ladders?

Help out Redwood Research’s interpretability team by finding heuristics implemented by GPT-2 small