Simpleton - your comment struck me as right on target except I would give this a positive value rather than a negative one. A lot of self help takes the form of akrasia-fighting; the question of course is whether it works. Amateur behavioral psychology would be one of the tools for separating effective from ineffective akrasia-fighting, yes?
The word amateur could perhaps use some re-valuing, especially in this context. The amateur, the non-professional, the person who wants to solve this problem for the personal benefit of enhancing his or her own decision making power, rather than for advancing within an established discipline or business: isn't this very category already subtly implied by the idea of rationality as a martial art?
Up with behavioral psychology + up with amateurs = up with amateur behavioral psychology.
The path that this conversation took is interesting, and may reflect the fact that this community is taking form around the sharing of ideas, rather than, say, collaborating on experiments. When I look back at the original post here, I see an invitation to privately attempt something, and then share the conclusions. This is valid and interesting but not the only possible approach. I've been talking to Wired, for which I often write, about doing a story on "rationality as a martial art," and the editors' immediate reaction was: if it is about real tools to become more rational, yes; if it is about people merely sharing and confirming their social identity as rationalists, no.
This isn't to say there is anything wrong or embarrassing about identity as a basis for community - that's probably the heart of the matter in most cases. But there are are other ways to hold communities together, and my original take on Less Wrong was that it also involved the application of ideas.
"Paranoid debating," at the last meetup, was actually quite instructive in this regard. I think it helped me become more rational to learn that I was the member of our group whose best intentions led the answer to be more wrong, and whose attempt to subvert actually pushed in the correct direction. I don't think this was through stupidity (the questions did not require much intelligence; only knowledge and realism about one's knowledge). Instead, several questions related to geography, and to places I had been, reporting on things that were unrelated to the questions. (Detroit: population; Alaska: size) Had I reported on these topics, I would have had real information. But since they were peripheral, I only experienced them passively, and formed vivid visual impressions. (Detroit: empty; Alaska: huge) My vivid visual impressions led to highly exaggerated estimates, and with no error checking in place, I argued for estimates that were wildly off.
Even though there were no stakes, I felt sorry - probably to just the right degree to make me remember the experience. I am still thinking about it several weeks later, and I am now more skeptical of something that research also confirms is a source of bias: visual impressions formed in the past without (much) rational analysis. This is not a scientific procedure, but an excellent example of rationality training in practice. It is not a private experiment, but a group collaboration.