I was interested to hear what more of the mech interp crowd thinks of this, so I am crossposting this here as well:
https://interpretability.blog/2025/07/22/interpretability-for-science-or-control/

...I will take this opportunity to give my personal and subjective experience of the “Actionable Interpretability” workshop at ICML 2025 and then just connect my thoughts with some of the larger trends in the field of interpretability.

I want to start off, however, with the end of this workshop, which had a panel like most. In one of the questions, the panel was asked:

Given that non-interpretability researchers often consider interpretability not that useful, how can we make our interpretability research more interesting to other fields, especially CV/NLP? In other words, are there interesting high-level directions which would allow us to better show the utility of our research?

One of the panelists gave the rather facetious answer that there was no need to convince anyone to join interpretability and that if you didn’t believe in the scientific understanding which interpretability promises, then “go add more GPUs or something”. This was of course met with a round of applause by the audience, before every panelist admitted it was more nuanced than that.

In this blog, I want to dig into that point a little harder, especially since the point of this year’s workshop was specifically designed to focus on the “actionability” of interpretability research, which seems to be in direct contradiction with this opinion.

In order to do this, I want to first bring up the recent direction pushed by Been Kim, an incredibly famous interpretability researcher and the opening talk of this workshop. She has recently pushed the direction of considering the “M” space of machine understanding and the “H” space of human understanding.

I think this is a very helpful framework to emphasize the goal of interpretability. In particular, there are some points inside of the “M – H” space which the machine ‘knows’ but the human does not. It is then the goal of interpretability to bring those points into the collective human understanding. This is exactly aligned with the goals of science.

In this most recent talk, Been Kim further stated that the direction going from M to H is what we would call “interpretability”, whereas the opposite goal of bringing the machine’s representation closer to a hum...

Posts

Wikitag Contributions

Comments