I remember reading that some activation stearing experiments used x100 size activation vectors. Is this correct? That would be much larger than the normal activation in that layer, right?

How does the model deal with this? If there is any superposition going on, I expect activation spill over everywhere, breaking everything.

If you amplify the activations in one layer, what effect has that on the magnitude of the activations in the next layer? If it's smaller, by what mechanism? That mechanism is probably an error correction algorithm. Or just some suppression aimed at not letting the network thinking about too many things at once, in order to keep down interference.

Does anyone have experience with activation stearing, and would be up for helping me out? E.g. aswer my questions, jump on a call, or help me set up my own experiments?