TL;DR: Neural networks can easily learn to regress cubic polynomials of type , but how do they do that? Results show that they internally access a variable , which is surprisingly close to the substitution used in the first step of the Cardano method for solving cubic polynomials. This project...
TL;DR Recent studies by Anthropic show that LLM features extracted via mechanistic interpretability fall into distinct categories, each with different properties. However, state-of-the-art auto-interpreters fail to account for this variety. In this article, I propose AIR (Auto-Interpretability Router). AIR is a new protocol that uses a sentence embedder to identify...
> TL;DR > > In this document, I propose baez a new feature label generation method that uses NLA explanations instead of activation examples. The codebase can be found here. > > In the experiment, the labels are generated both via baez and eleuther_acts_top5 are compared across various evals. The...