search quality: skimmed the abstracts search method: semantic scholar + browsing note that many of these results are kind of old
interesting to me but not what you asked for
https://www.semanticscholar.org/paper/The-learning-phases-in-NN%3A-From-Fitting-the-to-a-Schneider/f0c5f3e254b3146199ae7d8feb888876edc8ec8b https://www.semanticscholar.org/paper/Deceptive-AI-Explanations%3A-Creation-and-Detection-Schneider-Handali/54560c7bce50e57d2396cbf485ff66e5fda83a13 https://www.semanticscholar.org/paper/TopKConv%3A-Increased-Adversarial-Robustness-Through-Eigen-Sadovnik/fd5a74996cc5ef9a6b866cb5608064218d060d16 https://www.semanticscholar.org/paper/This-Looks-Like-That...-Does-it-Shortcoming...
Myself and some others did some work looking at the mutual information between intermediate layers of a network, and it's input here.
Recently I've been experimenting with recreating a neural network's input layer from intermediate layer activations.
The possibility has implications for interpretability. For example, if certain neurons are activated on certain input, you know those neurons are 'about' that type of input.
My question is: Does anyone know of prior work/research in this area?
I'd appreciate even distantly-related work. I may write a blog post about my experiments if there is an interest and if there isn't already adequate research in this area.