Statistical Summoning

I was thinking about Machine Learning in the context of the “multiverse probability mass” concept (often used by Eliezer) - which has us thinking in counterfactual universes, and some measure of how many of our alternate selves experience one outcome as opposed to another, and this led me to the thought that in that narrative frame, Machine Learning can be seen as a form of “mind summoning”.

We can imagine training a neural network by randomly perturbing its logical circuits, and making the random circuits that give the desired answer more likely (there is an equivalence between gradient descent and zeroth order random sampling[1]). There is an obvious analogy there, to “exploring locally” in the mind-multiverse until we manifest a mind that fits the outputs we desire.

Narratively, I wonder if there’s something there. Maybe it partly explains why training large ML models feels so much like summoning alien minds from alternate dimensions?

[1] Random Gradient-Free Minimization of Convex Functions. Nesterov, Y., Spokoiny, V.

Note: as the original post is 4 years ago, this is more of a note to self, the remains of an introspection process that was first meant to be private. I post it to challenge the part of myself that wants to stay comfortable in my beliefs, like a signal to myself that i need to write and be less afraid of sharing the results, to let my thoughts be criticised. So here goes:

i think I've experienced the same thing, at a particularly hard moment of my life. looking back, it didn't occur to me to spiritualize it, or pretend it cannot be rationally understood and communicated. The fact i could not introspect it fully was just a limitation of my own understanding, I had put too little conscious effort trying to explain exactly what it is. So it seems that I should read this post, be content about how it frames and even legitimizes something close to what I've felt, and move on. But instead, I notice that the post bothers me. Huh.

Here, I try to introspect: I notice that the Kensho post bothers me. No, it even almost makes me slightly angry. That's unusual, i think. Ok, let me try to understand why

Is it that it feels to me like giving up? My brain associates it 1 to 1 with moments in my life when people tried to tell me that things like rationality or morality cannot and should not be explained, that they are beyond science, and we should focus on the useful and on being good and faithful. Okay, that sounds about right, but I dont really associate those moments with anger, why the emotional reaction then?

I notice that when i read the post, inside my head, it comes out in the voice of the people who have tried to assure mystical authority over others, advocating intellectual laziness because it makes them sound wise and gives them status. And in the past these conversations had made me feel helpless, and frustrated, even angry. Closer!

What would I tell someone who said this to me? Maybe: did you, for example, have a realization that your sense of self is an illusion, which is useful for keeping you focused on optimizing your brain's value function, but which limits you when trying to make intellectual sense of the universe? Great! That can be described accurately, even though it's obviously foolish to expect the clear description alone would result in the same mental state being transferred to the reader (does anything work that way, anyways?), it is still useful to do so. In that situation, discussing the exact realization feels like progress in the common goal of rationality, whereas discussing the kensho (or qualia, or uniqueness in exact manifestation of the realization) aspect of it does not.

Or maybe you glimpsed some other deeper truth about the world, not in a way that you can accurately reason about what you know of this realization and how you know it, but in a way that it still affects your decision making, and you can notice that - a feeling of unexplainable improvement of one's subconscious compass, a first step towards understanding, a situation where your neural machinery encodes this new truth but higher level meta-analysers cannot yet introspect it. Fantastic! However, at that point not much can be done in terms of communicating its essence to others, since it is locked away from even being communicated to yourself.

Im sure that the author experienced what they say they experienced. I'm surprised, that that's what they would write about it, in LessWrong of all places.

It looks like part of the lesson here is that I must let go of my preconceptions, not everything that sounds wise and spiritual is bad, some of it must be limits of my own vision which I should keep an open mind towards. But also, some things in the original post do not pass my intuition's smell check, and following that smell I think there's something there too, this topic could have been approached differently.

I guess I may not see the use that crystallising this Kensho concept in the context of a rationalist tool, it seems more likely to do harm than be productive (maybe a 9:1 ratio?). Except maybe as a counter-example, a way to point towards dead ends? "Kensho-traps", feelings of spiritual understanding that are not empty of meaning, yet also not true understanding of a thing, but somehow give enough self gratification that they discourage further examination.