Rationality Verification Opportunity?

by beoShaffer1 min read15th Dec 201114 comments

0

Personal Blog

One of the challenges of rationality verification is that most people who are willing to contribute personal data for it are already familiar with the techniques involved.  This makes it difficult to tell if their performance on any form of rationality test is due to their training or their innate abilities.  Does the start of a new sequence present a way around this for that sequence's content?

I believe that it might, and will propose some ideas on how we can take advantage of these opportunities.  But first I would suggest that you try to think through the problem for yourself (I know this is slightly different from what is talked about in that post, but I think the principle holds).

 

 

Did you think through the general problem of rationality verification for new sequences before thinking of any solutions. Did you then think of your own solutions before getting your mind contaminated with mine?  If yes, good.  If no, not so good.

 

If we had good measures of general rationality that could be retaken by the same person multiple times without losing reliability we could simply ask LWers to take them at various intervals and see if they improved after reading the new sequence.  As that is not the case I suspect we would have to create specific measures for each sequence.  It seems that most writers have a decent idea of what benefits they expect people to gain from their sequences' so perhaps they could try to come up with specific measures for the things that their sequences are supposed to improve.  Then before running the sequence main sequence they could put out a call for people to complete these measures and send them in.  They could then collect the data again from people who have read the completed sequence, preferably after they have had enough time to practice the material, but not long enough to have had to many other life changes. The necessity and viability of having additional experimental controls will vary between sequences.  But I think we will generally be fine with a simple before and after picture. 

While there are some time and talent limitations I would be willing to help with creating the measures, collecting and interpreting the data and any other necessary steps.

I declare Crocker's rules on the content and style of this post.  This includes the title.

Personal Blog

0