Instrumental Rationality: Postmortem

by lifelonglearner 6 min read21st Oct 20175 comments

38


[In April, I set off to write a series of essays about instrumental rationality. Now that the project’s reached a pretty good stopping point, I’m looking back to see how my expectations and goals played out.]

Initial Goals:

[What I originally wanted the book to be like. My estimates vs reality]

Originally, I wanted to write something that would tie together all the current research on topics like motivation, planning, and habits. I felt like lots of LessWrong posts touched upon certain areas, like hyperbolic discounting, but there wasn’t a central place where it all came together.

I wanted a new central beacon to point people to when The Sequences didn’t quite fit.

I’d envisioned a sequence of essays which would give an overview of the latest developments in the field followed by concrete techniques, ala CFAR.

Here are some of the topics I’d originally wanted to cover:

  1. Willpower

  2. Attention

  3. Habits

  4. Behavioral Economics

  5. Motivation

When I began this project, I had Planning 101 under my belt, which had taken me about 20 hours to complete. As far as base rates went, it seemed reasonable to think that the other topics would take a similar amount of time.

Looking back, it feels a little silly to think that Past Owen thought he could take on five more of those 20+ hour chunk projects. That would have easily been 100+ hours, in addition to editing, compiling, and a bunch of additional grunt work which snuck up on me.

(What was I thinking?)

In the end, I only managed to write one more primer—Habits 101—which ended up taking about twice as long, 40 hours, for just the actual writing portion. Reading up on the articles took up additional time, probably another 10 hours or more.

Though I didn’t complete my initial vision, I actually did fairly well on my own estimates:

I’d given myself an internal completion date of the beginning of September, and I actually finished around that time. (Hooray!)

I’d also estimated that I’d write about 10,000 words of new content for the book. And I went about 50% more than that, writing about 15,000 words of actual new content. (Hooray!)

(Although the counterfactual for the new content prediction isn’t as good as it seems because I probably would have written some of those blog posts regardless of whether or not I also held the intention to make a sequence.)

The Finished Result:

[Evaluating the end product, some things which didn't make the final cut.]

So I ended up both finishing both on-target and on-time. How did the actual end product compare with my expectations?

Well, I’ve already pointed out how there was a lot of content I wanted to write which didn’t make it into the book. As for the content that did, here are the 9 sections that made it into the book:

  1. Introduction

  2. WTF Is Rationality?

  3. Starting Advice

  4. Planning 101

  5. Interlude 1

  6. Habits 101

  7. Interlude 2

  8. Attractor Theory

  9. Closing Disclaimer

Of the 9 sections, I think that WTF Is Rationality?, Starting Advice, and Attractor Theory come closest to the sort of “crystallization” I’d originally hoped for.

In the time between the original blog posts and the polished book essays, I’d had time to try explaining them to people in person. I think the experience helped me understand which things were important to focus on and what background knowledge I needed to assume. The revision and slow iteration of ideas helped me figure out which components were the important ones to stress.

So I think those three sections turned out the best.

As for the two 101 primers, I think they were too bulky, and I think a better choice would have been to sacrifice depth for breadth. That is to say, cutting the length of Habits 101 in half (or even two-thirds) in order to make way for a short primer on both Attention and Behavioral Economics would have likely been good.

Otherwise, I think it feels a bit jarring to switch from a short heuristic-y essays about ways to approach life to a deep academic dive into psychology.

As the sequence currently stands, I think it’s unbalanced in terms of the topics it goes over. Partially due to the length of the two 101 primers, I think there’s an overemphasis on planning and habits.

Overall, I think it pushes the “master yourself” mindset over the “become one with yourself” mindset too hard, which I ended up writing about in the Closing Disclaimer.

Perhaps unfortunately, I’m also fairly confident that most people who read it won’t get much benefit out of it. (More on this in Evaluating Impact.)

Starting out, I also had some big hopes for the actual format of the book.

I think that most writing doesn’t do a good job of helping the reader chunk the information. That is to say, the information trying to be conveyed is often much clearer in the author’s mind than in the readers’. I claim this is often because the author has their own way of mentally structuring the information which they neglect to share during the actual writing.

Frustrated with this, I’d originally wanted to include several design features for the sequence to help with understanding:

  • Suggested Exercises at the end of every chapter, focused on developing practical skills.

  • Unique formatting, perhaps a different font color, for paragraphs which contained examples (to visually differentiate them from the rest of the text).

  • Visual outlines of how points connected in each essay. (You can see some simple prototype attempts in Habits 101.)

In the end, the only one that made it to the final book are the bracketed summaries in italics that precede each section.

I think that conveying ideas in general is actually rather quite difficult. Cooperation is required on both the part of the author and the reader, and even an engaging writing style (or cool design tricks) can only do so much.

Writing Time:

[Reflections on the actual writing process.]

I ended up having less time to write the entire book than I originally thought.

No surprise there.

(“You don’t plan for disaster. Disaster plans for you.”)

Part of the reason was because I was physically unavailable on account of my being at Google doing CS things for three weeks.

The other part was that I found myself mentally incapable of writing quality content for long bouts of time.

The roughly 100 hours I put into writing this book was scattered over about 4 months. That ends up being less than an hour a day, though the distribution of hours wasn’t uniform. When it came to writing this book, I found that certain times yielded far more productivity than others.

Also, looking back, it feels like the entire duration of time was necessary, even if most of it wasn’t actually spent writing. It feels like I just needed time in between writing sessions to let my mind do its thing under the hood, subconsciously. I’d write for about an hour, wander about the room (or do something else for an hour), and then I’d return to writing.

This seems like a specific instance of the general principle that breaks aren’t just a “fun” activity, but are actually a requisite for good work to happen.

So it was less about getting enough free time in a chunk, but more about getting enough of the right kinds of time, or something like that. For context, I think that one thing people overlook when considering making tradeoffs involving time is the nature of the time they’re gaining or losing.

For example, if you’re able to save ten minutes off your commute, that roughly translates to having ten additional minutes to spend at work, which might not be worth much. In contrast, if you’re able to extend your lunch break by twenty minutes, that could be enough time for a noontime nap, which might be very valuable.

Against the Incentive Gradient

[Writing longform has certain drawbacks.]

When I was working on this project, a friend pushed me to consider writing for a larger group like BuzzFeed or ClearerThinking. They argued that even though the type of content engagement I’d get from readers would be less, the net increase in audience size would mean that the aggregate impact would overall be larger.

The argument seemed good, but I ended up sticking to my original plan to write in longform.

(Longer exploration on this topic in The Best Self-Help Should Be Self-Defeating.)

Final Lessons:

[What were my concrete takeaways?]

There were several things I learned as a result of undertaking this project:

  1. My internal estimates for my task completion time and writing rate are fairly well-calibrated.

  2. I can make graphics of a quality I’m happy with at a rate of about 1 graphic an hour.

  3. Experience and understanding of what research looks like. (EX: Filtering through papers to find promising things, writing summaries, simplifying at the right level, etc.)

  4. Getting constant feedback is both important for my ability to continue projects, as well as improve the quality. I didn’t get enough of it this time around.

  5. I had, overall, still underestimated the difficulty involved in writing and editing a book-style project.

The biggest one, though, was the burning question I had when setting out on this project: “Why the hell hasn’t anyone tried to make this type of freely available rationality handbook before?”

The answer, I think, looks something like this:

If you’re writing self-help content, the question of who the audience is inevitable. After all, at some point, someone’s supposed to be reading the content. And in my current understanding of people, it seems like there’s roughly two categories of readers (gross oversimplification alert!):

One, if you’re already smart and self-sufficient, then you’re already combing all the coolest blogs and books for insight. You’re a hyper-scholar who has either read and gotten value out of what I have to offer, or you’ve already got even more sophisticated models.

Two, if you’re the sort of insight junkie who’s looking for the next 5 minute article on how to restructure your workflow, then you may miss out on the actual good stuff. If you’re always only looking at the small insights, then the ontological lens of rationality (which is what I really want to illustrate), is largely lost.

So I guess one strong reason for why the sort of rationality handbook I was envisioning hasn’t happened is because it doesn’t really help out either of the groups.

The target audience is someone who’s a little mixed up in their stages of intellectual development.

I’m reminded of how I felt after talking about the book Gödel, Escher, Bach with some friends studying computer science. It’s a book which I very much enjoyed. However, none of them had been impressed with it, and they found it largely pedantic, taking too long to get to some basic insights.

Most people who read GEB, I suspect, either get lost in the dizzying array of cultural references and come out with a messy interpretation, or are already with maths and computer science such that the key points fall flat.

When I read it, I was in this weird spot where I didn’t know any computer science, but I also looked past the references, and I found something exciting that encouraged me to dive deeper.

++++

With this instrumental rationality sequence, I’d wanted to write something for other people to benefit from. In a funny sort of way, though, I guess I really did just end up writing a book for myself.

38