I was reading your article and I had this wild "what if" thought that I would like to share. Who knows what a spark like that could ignite, right? :)
What if... during the training we also train a second more constrained "explainer" DNN designed to infer the neuron activations of the main DNN being trained? If we find a way to project the internal representations into a more compressed latent space, maybe this secondary NN could learn how to represent this high-level abstraction that represents the NN's internal mechanics. These "explanatory embeddings" could be used to correlate different neural pathways and help us to understand what types of abstractions the neural... (read more)
Nice work!
I was reading your article and I had this wild "what if" thought that I would like to share. Who knows what a spark like that could ignite, right? :)
What if... during the training we also train a second more constrained "explainer" DNN designed to infer the neuron activations of the main DNN being trained? If we find a way to project the internal representations into a more compressed latent space, maybe this secondary NN could learn how to represent this high-level abstraction that represents the NN's internal mechanics. These "explanatory embeddings" could be used to correlate different neural pathways and help us to understand what types of abstractions the neural... (read more)