What sort of substantiation were you looking for? The article pretty clearly states that the claims about the effects of the camp were based on exit surveys, and that the impact of the camp is demonstrated by the projects the camp grads are now working on. You could debate whether those are good measures, but we don't exactly have better ones.
I agree it would be nice if we could come up with standardized tests for rationality and then test whether camp attendance improves scores, but even if this were possible it hardly seems the best conceivable use of SIAIs resources.
Well, I'd want to know what thes supposed projects are, what progress they've made, how well people are sticking to them, etc. It's not much help to mention self-assessed exit surveys, given that people are remarkably poor judges of their own skill. The self-assessments would really only measure how much fun the participants had, which was not the standard it claimed to judge itself by.
They could also post improvements on tests -- since there were specific skills taught, surely there's an easy before/after they can come up with. For the males (given what I remember about what lukeprog was going to teach), they can report (anonymized) "dating effectiveness improvement".
For a smashing success, there's remarkably little I can see to distinguish it from non-success. (And when such evidence does come, it hope I'm not criticized for asking for it.)
Here at the SI blog. Thanks again to all who made it an incredible minicamp!