What is self-inquiry? Self-inquiry is an esoteric spiritual practice geared towards the deconstruction of the self. Why would anyone want to do that? What makes this practice spiritual? Why can it be found in the core of many spiritual traditions? What evidence of its effectiveness do we have?
First, why do I want to deconstruct the self? If I carefully observe my thoughts I can see that most problematic thoughts are self-referential ones. To put it simply, thoughts that are built around I/me/my. They make me anxious and afraid. In most cases without any good reason.
To check that this is so, one can conduct the following experiment (as described by Gary Weber[1]). Get yourself a piece of paper and a pencil. Draw two buckets: one with I/me/my thoughts, another without I/me/my thoughts. Sit quietly for five minutes and observe your thoughts. Each time a thought arises make a mark in the appropriate bucket. Count your thoughts in the end.
One remark is that some thoughts may have a hidden and not obvious I/me/my structure. They should be counted in the first - I/me/my - bucket.
If your thinking is like mine, you will get around 90-95% of thoughts in the first bucket. In this way one can get an insight into what's going on in one's head while being idle.
Further one may ask, "Ok, the majority of idle thoughts are all about me, but why would I want to get rid of them?" Again, if your thinking is like mine and you observe the content of your self-referential thoughts you will discover that most of them generate anxiety and fear in some way or another. They also generate grasping and attachment. And grasping and attachment are the causes of suffering. That's Buddhism 101[2].
The next line is simple. If I/me/my thoughts generate suffering, how can I get rid of them? And the answer to that is by deconstructing the "I" one can get rid of all problematic self-referential thoughts (yes, it's possible, it's called the state of nonduality[3]). Which would free one from the constant internal dialogue that almost all of us have.
So in the end, the simple answer to the question, "Why would anyone want to do that?" is to get rid of suffering. Pretty amazing if you ask me. And how does one go about it? By the process of self-inquiry.
What makes this practice spiritual? As I see it, the main reason that that practice may be considered spiritual is twofold. One aspect is that in the beginning one has no evidence that it will work and no evidence that it's at all possible to get rid of self-talk. So one starts the process with the faith that it will work. Another aspect of it is trust in another human being, who has been through it till the end, and now shares what she/he has found. (At least, it was so until very recently. Everything has changed since we started putting our brains through fMRI and seeing the evidence). So both aspects are really reducible to trust. That is what makes (made) that practice spiritual.
Why can it be found in the core of many spiritual traditions? It is indeed so that in their esoteric core most spiritual traditions work with the redirection of attention from the outside to the inside and making it abide in awareness. And that practice may be found sometimes openly, sometimes only transferred to "the worthy disciples" not to dilute "the purity of the teachings". So one has to know what one is looking for in order to discover it in all major traditions. But it's there in one form or another.
What evidence of its effectiveness do we have? In short, the resulting state of nonduality which follows that practice (if it's seen to the end), can be measured by fMRI as having no self-referential thoughts and by reduced stress levels. It's something that Gary Weber discusses in detail in his lecture on Myths about Nonduality and Science[3].
But how do we know that that particular practice is optimal in getting rid oneself of suffering. Here, I confess, we still have to trust another human being's words and a word of mouth for that. We haven't proven it yet. But we have the words of those people who have stepped onto the other shore and describe the shortest route to get there. However, we've already proven that in order to shut down the ruminating network (the DMN[4]) we have to do some task (and switch to the Task Positive Network[5]). And self-inquiry is fit for that. Of course, it doesn't prove that it's optimal. There are many types of meditation geared toward the same end.
One more important remark concerning the practice. It doesn't mean that one has to get all the way to the end to get rid of suffering. As some of it will be falling apart during the performance of practice itself. So it makes it a universal tool to handle problematic thoughts and suffering on an ad hoc basis.
With all those questions out of the way, let's proceed to the core of the practice as it was brilliantly put by "the sage of the 20th century" Ramana Maharshi:
When other thoughts arise, one should not pursue them, but should inquire, "To whom do they arise?" It does not matter how many thoughts arise. As each thought arises, one should inquire with diligence, "To whom has this thought arisen?" The answer that would emerge would be "to me". Thereupon if one inquires "Who am I?", the mind will go back to its source; and the thought that arose will become quiescent. With repeated practice in this manner, the mind will develop the skill to stay in its source.
… The thought "Who am I?" will destroy all other thoughts, and like the stick used for stirring the burning pyre, it will itself in the end get destroyed.
-- Ramana Maharshi, Who am I?[6]
Another Zen master Bassui Tokushō expressed it in the following way (self-inquiry is called koan practice in Zen tradition):
If you want to realize your own Mind, you must first of all look into the source from which thoughts flow. Sleeping and working, standing and sitting, profoundly ask yourself, "What is my own Mind?"
… What kind of master is it that this very moment sees colors with the eyes and hears voices with the ears, that now raises the hands and moves the feet?
… "My body is like a phantom, like bubbles on a stream. My mind, looking into itself, is as formless as empty-space, yet somewhere within sounds are perceived. Who is hearing?"
… While you are doing zazen neither despise nor cherish the thoughts that arise; only search your own mind, the very source of these thoughts.
… You must only become the question "What is this Mind?" or "What is it that hears these sounds?"
… Within yourself you will find no "I", nor will you discover anyone who hears. This Mind is like the void, yet it hasn't a single spot that can be called empty.
… Only when you have completely exhausted the questioning will the question burst; now you will feel like a man come back from the dead. This is true realization.
-- Philip Kapleau, Three Pillars of Zen[7], Bassui's Sermon on One-Mind
And Tibetan Buddhist master Longchen Rabjam expressed it in the following way:
Six pieces of consummate advice relate to what is ultimate:
If you wish to realize the view, search for the one who realizes; you will find no such thing, and so are free of the trap of the four conceptual extremes.
If you wish to cultivate meditative absorption, search for the meditator; you will find no such thing, and so contemplate the fact that mental stirring has no substantial nature.
If you wish to engage in a mode of conduct, search for the one who engages in conduct; you will find no such thing, and so are free in your natural, undifferentiated state.
If you wish to realize the fruition, search for the one who realizes; you will find no such thing, for the three kayas are perfect within you.
If you wish to accomplish enlightened activity, search for the one who accomplishes; you will find no such thing, for activity is spontaneously accomplished without having to be deliberately carried out.
If you wish to traverse the spiritual paths and levels, search for the one who makes the journey; you will find no such thing, so direct yourself to your natural state, the way of abiding.
This is my advice concerning the consummate heart essence: please understand that everything is the miraculous display of awareness!
-- Longchen Rabjam, The Precious Treasury of Pith Instructions[8], Consummate Advice
What concerns how much and when to practice. There is bad/good news here. Almost all teachers unequivocally stressed out that the practice of self-inquiry has to possess you and take all of your free time to reach the definitive moment of liberation. However, mileage varies. Some people can awaken with a busy schedule and few hours of practice per day (e.g. Dominic's case[9] and Gary Weber himself[10], although he did tons of practice, it wasn't divorced from his life).
But most importantly one doesn't have to dedicate all her/his time to self-inquiry and go all the way to the end to get the benefits from practice. So every little bit works and helps.
Gary Weber, Myths about Nonduality and Science.
Ramana Maharshi, Who am I?
Philip Kapleau, Three Pillars of Zen.
Longchen Rabjam, The Precious Treasury of Pith Instructions.