Good post. I like how you explained both the technique and the process that you used to develop it.
I see another potential benefit in estimating VoI. Asking myself, "Does any state of knowledge exist that would make me choose differently here?" bypasses some of my involuntary defenses against, "What state of knowledge would make me choose differently here?" The difference is that the former triggers an honest search, while the latter queries for a counterfactual scenario but gives up quickly because one isn't available.
I have a slightly less careful method:
I feel like I want to be sure I'm making the best decision, no matter what.
As the effort of becoming more certain that I am definitely making the best choice becomes unpleasant, I try to escape by convincing myself that the satisfaction of resolving the issue exceeds even my optimistic projection of what improvement further optimization effort will bring.
The first is a personality flaw. The second is a perhaps unnecessarily painful way of managing it. But it seems natural to me that the burden of proof is on my pain-avoiding rationalizing self to spare me conscious deliberation. Since I'm in no way protected against over-deliberation that feels good, I suppose if I route around the flaw only for low-value-of-thinking/searching cases, I'll be better off as well as happier.
I liked your characterization of "feels big" vs. "nagging doubt" hesitations, and your suggestion of trying to remember sensations/feelings that happened in advance of losing a gamble (although that way leads to superstition as well). "Visualize" is a good suggestion - although of course we're biased toward what visualizes most vividly for us. And with enough experience, routine decisions are made very cheaply - without visualizing anything.
I liked that you tried to systematically break down how this works exactly, but considering how wrong our intuitions about the thinking we do on very short time scales is (like how some part of me decides to move before my conscious mind is aware of it, yet it sure feels like I've consciously decided to do so), I'm not sure how useful such introspection is.
Self-experimentation with some exterior criteria by which evaluating your performance or at least something outside of easily corruptible human memories recording what is going on might be better.
Scientific studies directed into testing such techniques would obviously be even better. Perhaps some that are relevant exist. Perhaps they don't. Too bad we don't have an Institute of Applied Rationality (acronym comes out awkward, heh) that could do rigorous psychological testing for such things. Or at least get a few random LW people to under supervised conditions try out stuff like this.
Some effort towards empiricism is I think better than none. The VoI might however not high enough for this to be a good idea (irony).
This post started off as a comment to Vaniver's post Value of Information: Four Examples. This post also heavily builds on Eliezer's post The 5-Second Level. The five second level is the idea that to develop a rationality skill, you need to automatically recognize a problem and then apply a stored, actionable procedural skill to deal with it, all in about five seconds or so. In here, I take the value of information concept and develop it into a five second skill, summarizing my thought process as I do so. Hopefully this will help others develop things into five second skills.
So upon reading this, I thought "the value of information seems like a valuable concept", but didn't do much more. A little later, I thought, "I want to make sure that I actually apply this concept when it is warranted. How do I make sure of that?" In other words, "how do I get this concept to the five second level?" Then I decided to document my thought process in the hopes of it being useful to others. This is quite stream-of-consciousness, but I hope that seeing my thought process helps to learn from it. (Or to offer me valuable criticism on how I should have thought.)
First off, "how do I apply this concept?" is too vague to be useful. A better question would be, "in what kinds of situations might this concept be useful?". With a bit of thought, it was easy to find at least three situations, ones where I am:
1. ...tempted to act now without gathering more information, despite the VoI being high.
2. ...tempted to gather more information, despite the VoI being low.
3. ...not sure of whether I should seek information or not.
#3 implies that I'm already reflecting on the situation, and am therefore relatively likely to remember VoI as a possible mental tool anyway. So developing a five-second level reaction for that one isn't as important. But in #1 and #2 I might just proceed by default, never realizing that I could do better. So I'll leave #3 aside, concentrating on #1 and #2.
Now in these situations, the relevant thing is that the VoI might be "high" or "low". Time to get more concrete - what does that mean? Looking at Vaniver's post, the VoI is high if 1) extra information is likely to make me choose B when I had intended on choosing A, and 2) there's a high payoff in choosing correctly between A and B. If at least 2 is false, VoI is low. The intermediate case is the one where 2 is true but 1 is false, in which case it depends on how extreme the values are. E.g. only a 1% chance of changing my mind given extra information might sometimes imply a high VoI, if the difference between the correct and incorrect choice is a million euros, say.
So sticking just to #1 for simplicity, and because I think that's a worse problem for me, I'd need to train myself to immediately notice and react if:
(I'm about to do something at once) AND [(Extra information might change my mind AND changing my mind matters) OR (extra information is pretty unlikely to change my mind AND changing my mind matters a lot)].
That's a relatively complicated conjunction to evaluate in a second, without conscious notice. But my mind does carry out far more complicated evaluations without my conscious notice, all the time. So the question is not "is it possible to learn this", but "how do I learn this"?
Fortunately, having gotten to this point of analysis, it's pretty clear that I already have some mental sensations corresponding to some of those criteria. Noticing the (extra information is pretty unlikely to change my mind AND changing my mind matters a lot) part comes naturally: decisions with big consequences naturally make everyone hesitate. That natural hesitation helps in recognizing both #1 and #2 situations: whenever I feel the hesitation involved in a big decision, I can ask myself, "what's the VoI involved in this, am I hesitating too little or too much?". So there's the initial trigger for those kinds of situations. I had already learned the "if you can't decide, flip a coin" heuristic from earlier, and sometimes applied it, so in effect I already had one VoI-related 5-second procedure in place.
What if the consequences for an action are big, but I don't realize that? Recognizing that probably requires its own set of 5-second procedures, so I'll let that be for now.
Then there's the (Extra information might change my mind AND changing my mind matters) condition. That's trickier, but I also have a related trigger for that - that little nagging doubt, a feeling of something not being quite right. That's also something that mind automatically activates in some situations, without me needing to consciously think about it. It might be good idea to develop a five-second procedure for training this instinct. Perhaps I could train it by stopping each time when I realize I've been wrong in something that matters, and asking whether I felt the doubt and whether I should have. But for now, it's enough to know that I have such a natural reaction that I can use.
So now I have two triggers for a five-second reaction - the hesitation upon a big decision, and the little nagging doubt. Great. What's the procedure I need to automatically associate with these feelings?
The first one that comes to mind is "trigger a conscious VoI evaluation". I'm strongly tempted to just leave it at that. I've already written a lot but, in retrospect, only said things that feel obvious. And there's a certain cogsci/compsci geek charm in putting it that way. System 1 notices a possible problem and flags it for System 2 to evaluate. A computer device driver fails to fix a read error, and passes the issue one layer up to the operating system. That's how it's supposed to go, right?
But no. How likely am I to actually "do a conscious VoI evaluation"? I'm lazy, just like everybody else. Just realizing that I'm supposed to do something doesn't mean that I'll actually do it. If I can have a lower layer of my mind fix the problem without bothering the higher layers, I'm much more likely to actually get the problem fixed. Besides, I don't even know what a "conscious VoI evaluation" means, yet. I need to put it into explicit steps, and if I can do that, I might be able to automate it, too.
So. I was going to do something, but it's a big decision so I hesitate, or it isn't that big of a decision, but I feel a little doubt anyway. What do I do next?
I skim Vaniver's original post, looking for direction, and then realize that I'm being too abstract again. I need a concrete example, and then I can try to generalize a pattern from that. When have I felt that hesitation or doubt lately?
The answer: in various online and offline discussions. I've been writing something that reveals possibly embarrassing personal details about myself, or opinions that I might not want everyone to know that I hold. I've felt the doubt of the question, is this something that I should say in the open, or should I hold it back?
This looks like a good candidate to analyze, especially since I haven't really analyzed the question in very much detail in those situations. I've just gone on a gut instinct and held back, or decided to just act anyway despite my doubts. What should I do in that situation?
Again, I realize that I'm maybe too abstract, and should go for more concreteness. Let's suppose that I had a sexual interest in sheep. (I do not, for the record - this is just an example.) Further suppose I was having a public conversation with somebody over at Google Plus, and I was going to mention this, but then I hesitated. Do I really want the whole world to know about me and sheep? What would be the right course of action, here?
First to evaluate the possible consequences of me saying this versus me not saying this. Maybe all the people who I need to care about are enlightened enough that they don't fault a person for his sexual interests, neither on an intellectual nor an emotional level. And maybe getting to tell others of this makes me relieved. Now I don't need to feel that this is something shameful that must be kept as a secret anymore.
On the other hand, it could be that this knowledge spreads and I'm widely ridiculed and made fun of. Or maybe it won't be that bad, but the person who I'm talking with will feel awkward about this unexpected revelation and that will damage our relationship. Or maybe the outcome is ambiguous, and I'll spend the rest of my life worrying whether the knowledge will spread and embarrass me one day. The stakes are clearly high here. I might rid myself of the feeling of carrying a shameful secret with me and realize that nobody really finds it to be an issue after all, or I might pretty much ruin my life forever. So choosing correctly matters.
If I found out more, is that likely to alter my estimates of the probability of the various consequences? Well, that depends on how confident I'm feeling in my model of other people and the average repulsiveness of sheep fantasies. So I have to evaluate that. Humans are often surprising, groups of humans are even more surprising, and the typical mind fallacy is often strong. So I don't think I have high confidence in my model of the consequences. The VoI thus seems high here. So I hold back the revelation for now, perhaps discreetly gauging my conversational partner's reactions to such topics, and starting to collect such information from other people as well.
Concrete example analyzed. Time to go back to the abstract. What did I do here? First, I stopped to consider the various potential consequences. Then I asked myself how sure I felt in my model of those different consequences happening. When I realized that I was both unsure, and that a wrong decision might be costly, I decided to hold off for now and gather more information. This, as well as recalling the definition of VoI, suggests the following procedure:
1. Notice a momentary doubt or hesitation when about to do something.
2. Pause to visualize various consequences of the action.
3. Contrast the most important consequences and ask, "would it be a big deal for me if one of these happened and the other wouldn't"?
4. If yes, then for any such major consequence, ask "how surprised would I be if this actually happened"?
5. If there are several possible consequences that wouldn't be very surprising, and it matters which one of them happens, ask "is gathering more information likely to make at least one of these outcomes seem more surprising"?
6. If yes, hold off and gather more information. If not, just do what the current expected utility estimate says is the best action, since it isn't likely to change. If the expected utilities are close, just flip a coin.
I'm still not entirely sure if that's close enough to the five-second level - it seems like there's still a lot of conscious evaluation involved. But at least I now know the steps needed for the conscious operation. Now that I've written this out, it seems so obvious as to not be worth posting about... but then, no matter how obvious it feels like now, I haven't been consistently doing that. So this was probably worth writing down anyway.
This exercise also suggested two new five-second techniques that I might want to look into. First, recognizing big decisions that don't seem like big decisions at first. Second, if I've screwed up when I should have felt that hesitation, teaching myself to feel that hesitation the next time. Also, writing this down as I went along was really useful - I don't think I would have gotten anywhere this far if I hadn't. A final lesson was that moving towards concrete examples was crucial in developing the skill. I made pretty much no progress with questions like "what sort of a reaction should my triggering sensation actually trigger" or "what should I do in the general situation of being unsure of whether to say something", but when I moved into concrete examples, the answers were quick to come to me.