Rationality Exercises Prize of September 2019 ($1,000)

by Ben Pace 2mo11th Sep 20198 min read18 comments


I'm giving out $1,000 of prize money for the best exercises submitted in the next two weeks on a topic of interest to the LW community. I'm planning to distribute $1,000, with $500 of that go to the first place.

To submit some exercises, leave a comment here linking to your exercise(s) by midnight at the end of Friday 20th September PDT (San Francisco time), and I'll announce the winners by the Friday two weeks later (give me the time to try a bunch out). You're welcome to post them as a LW post, on your shortform feed, or privately link them to me in a PM if you want, though I'll be publishing all the entries that win a prize.

Why exercises?

I'd like to be to practice using ideas, and to know whether I actually understand them.

I want to concretely practice the art of rationality (and other arts), but I don't have many natural affordances to do that. If people added exercises to their posts, I think that I'd do them. I do sabbath-like recovery days, and I have a weekly session with Jacob Lagerros where we work on problems in the HPMOR-recommended book Thinking Physics, both of which I find exceedingly valuable. I'd love to spend more time playing with other ideas people put forward on LW.

I also think it's surprisingly common for me and a friend to achieve a double illusion of transparency where we're both using a concept or phrase in conversation, but actually have a very different referent in mind. I think small tests and checks can zoom in surprisingly quickly on miscommunication.

So I'm running the prize to get some exercises, for me and for others on LW who want to try them out.

I could talk more about why exercises are valuable, but a lot of my thinking here is downstream of reading the book Thinking Physics, so I'd rather just let its author, Lewis Carroll Epstein, speak instead. (This is from the opening of the book, all formatting is original.)

The best way to use this book is NOT to simply read it or study it, but to read a question and STOP. Even close the book. Even put it away and THINK about the question. Only after you have formed a reasoned opinion should you read the solution. Why torture yourself thinking? Why jog? Why do push-ups?
If you are given a hammer with which to drive nails at the age of three you may think to yourself, "OK, nice." But if you are given a hard rock with which to drive nails at the age of three, and at the age of four you are given a hammer, you think to yourself, "What a marvellous invention!" You see, you can't really appreciate the solution until you first appreciate the problem.
What are the problem of physics? How to calculate things? Yes - but much more. The most important problem in physics is perception, how to conjure mental images, how to separate the non-essentials from the essentials and get to the hear of a problem, HOW TO ASK YOURSELF QUESTION. Very often these questions have little to do with calculations and have simple yes or no answers: Does a heavy object dropped at the same time and from the same height as a light object strike the earth first? Does the observed speed of a moving object depend on the observer's speed? Does a particle exist or not? Does a fringe pattern exist or not? These qualitative questions are the most vital questions in physics.
You must guard against letting the quantitative superstructure of physics obscure its qualitative foundation. It has been said by more than one wise old physicist that you really understand a problem when you can intuitively guess the answer before you do the calculation. How can you do that? By developing your physical intuition. How can you do THAT? The same way you develop your physical body - by exercising it.
Let this book, then, be your guide to mental pushups. Think carefully about the questions and their answers before you read the answers offered by the author. You will find many answers don't turn out as you first expect. Does this mean you have no sense for physics? Not at all. Most questions were deliberately chosen to illustrate those aspects of physics which seem contrary to casual surmise. Revising ideas, even in the privacy of your own mind, is not painless work. But in doing so you will revisit some of the problems that haunted the minds of Archimedes, Galileo, Newton, Maxwell, and Einstein. The physic you cover here in hours took them centuries to master. Your hours of thinking will be a rewarding experience. Enjoy!

What does this look like?

Here are great exercises that have been on LessWrong in the past.

In my primer on Common Knowledge, I opened with three examples and asked what they had in common. Then, towards the end of the post, I explained my answer in detail. I could've trivially taken those examples out from the start, included all the theory, and then asked the reader to apply the theory to those three as exercises, before explaining my answers. There's a duality between examples and exercises, where they can often be turned into each other.

But this isn't the only or primary type of exercise, and you can see many other types of exercise in the previous section that don't fit this pattern.

What am I looking for in particular?

While I'm open to most possible subjects, let me add one operational constraint: it should be an exercise that more than 10% of LessWrong commenters can understand after reading up to one-to-three posts you've specified, or after having done your prior exercises. As a rule I'm generally not looking for a highly niche technical problems. (Though it's fine to assume people have read any curated LW sequence.)

I asked Oli for his thought on what makes a good exercise, and he said this:

I think a good target is university problem sets, in particular for technical degrees. I've found that almost all of my learning in university came from grappling with the problem sets, and think that I would want many more problem sets I can work through in my study of both rationality and AI Alignment. I also had non-technical classes with excellent essay prompts that didn't have as clear "correct" answers, but that nevertheless helped me deeply understand one topic or another. I think both technical problem sets and good essay prompts would be great submissions for this prize, though I'd encourage providing at least suggested solutions (probably best posted behind spoiler tags).

(What are spoiler tags? Hover over the text of this post to read the black box below.)

This is a spoiler tag! To add this to your post or comment, see the instructions in the FAQ that's accessible from the frontpage on the left-menu.

(Also see this comment section for examples of lots of people using it to cover their solutions to exercises. Also to see the level of demand for exercises on LW.)

I'm interested in exercises that help teach any key idea that I can't already buy a great textbook for, although if your exercises are better than those in most textbooks, then I'm open to it too.

I think technical alignment exercises will be especially hard to do well, because many people don't understand much of the work being done in alignment, and the parts that are easy to make exercises for often aren't very valuable or central.

I think that it's often easier to build exercises for very explicit, legible concepts (e.g. things that look more like math), and while that's really valuable, I'm also really excited about exercises for other ideas too.

Examples of things I think could have exercises

Definitely exercises for any curated post or curated sequence on LessWrong. I've taken a look through our curated posts, here are a few I think could really benefit from great exercises (though the tractability varies a lot on these).

Curated Sequences

Here are examples from Curated Sequences.

Curated Posts

Robin Hanson has masses of brilliant ideas that I've not got the time to mine. Some that comes to mind are his more recent posts on automatic norms that I think could become some really great exercises.

Some of Nick Bostrom's ideas would be excellent too, like the unilateralist's curse, or the vulnerable world hypothesis, or the Hail Mary approach to the Value Specification Problem.

If you leave a public comment describing what sort of exercises you might want to try creating, I will try to reply with my best guess on whether it can be a good fit for this prize.

I'll repeat: To submit exercises, leave a comment here linking to your exercise(s) by midnight at the end of Friday 20th September PDT (San Francisco time), and I'll announce the winners by the Friday two weeks later (give me the time to try a bunch out). You're welcome to post them as a LW post, on your shortform feed, or privately link them to me in a PM if you want, though I'll be publishing all the entries that win a prize.

I look forward to trying out your exercises.

Added: Prizewinners announced in this comment below.