TL;DR: I organised a series of afternoon applied rationality workshops for the Cambridge Effective Altruism group based on some core CFAR classes. These went much better than I expected them to, and seem to have added long-term value to participants. My goal in this post is to share my resources and lesson plans, my thoughts on teaching applied rationality well, my attempts to distill the concepts into key ideas & mental habits, and to convince other people to organise similar workshops!
(Disclaimer: This was inspired by CFAR material, but is very much all my interpretation and framings. Any problems with the material are likely from me, not CFAR)
I went to a Centre For Applied Rationality (CFAR) workshop last year and found this a really valuable experience. I especially found that a few techniques stuck well with me and became valuable parts of how I thought. So, to help consolidate these and to share them with others, I organised some afternoon workshops for the Cambridge EA group based on my favourite classes. I think these were extremely successful and added more value than I expected to the participants. So this post is my attempt to write up a retrospective on the workshops, what I think was valuable about them, and what I learned from them.
I taught four workshops:
The format was fairly different from the standard CFAR workshop (90 minute afternoon workshops every 2 weeks, rather than an intense 4 day workshop), which I expected to make it much harder to have an impact. But, based on a followup 2-3 months later with participants, I found that some insights had stuck, and were being used regularly. A significant component of this was that I focused on distilling the techniques down to the key ideas, and useful mental habits. These seemed to stick well with some people, without requiring much followup effort.
Overall, I feel very happy with the impact, and I think afternoon classes are low cost to run (and seem to transfer acceptably to remote). I don’t think a CFAR-style framing of rationality resonates with everybody, but there are enough people who found it high value that I think this was super valuable on average. So I would be very excited to see more people try to run classes like these!
I’d be especially excited to see more EA student groups running them I expect improving the long-term effectiveness of young EAs to be very high leverage, and a significant way for a local group to add value. I’d also expect it to be valuable to the organisers, I found writing and running these workshops extremely helpful for understanding the ideas more deeply myself, and for improving at teaching! And I’ve found myself applying these ideas much more widely in my life.
I have a few different goals with this post. As a result this post ended being pretty long, so I’ve tried to write each section to be self-contained, and to indicate which sections different people might find interesting:
Format: These were 90 minute afternoon classes on weekends, mostly aimed at student EAs (late teens/early twenties). The target audience were people with a prior interest in EA, rationality and optimising their life, but without much specific experience of CFAR techniques.
The following is a rough summary of the key takeaways and structure of my productive disagreements workshop (based on CFAR’s Double Crux class). I’ve tried to go into detail on Pedagogical-Content Knowledge (PCK): knowledge about the topic, how students tend to engage with it, and how to teach it well. Eg examples that worked well, common misconceptions, subtle nuances to emphasise, etc. I think PCK is really useful to teach the classes well, but also very useful to understand the ideas more deeply yourself, even if you don’t intend to teach them. In the interests of space, I’ve put similar sections for my workshops on planning, habits and systems in Appendix A.
I tried to heavily frame each workshop around building mental habits and reflexes. Ie, rather than the point being to learn a long, effortful algorithm, breaking the algorithm into bite-sized steps, learning the cues for when each step is relevant, and learning to bring them up in the moment. I think this is a skill best trained with TAPs. And based on the long-term feedback, this was an extremely successful approach! Few participants put in much effort to practice the techniques, but several managed to absorb these mental habits
People generally enjoyed the workshops, and when asked immediately afterwards gave highly positive feedback. I think the main source of impact is whether people absorb these techniques in the long-term, so I followed up 2-3 months after the workshops, and asked for qualitative feedback on how well the techniques had stuck. I heard back from about 85% of participants.
My best attempt to summarise this data was to loosely categorise people into neutral (no real impact), moderate successes (some long-term benefit) and strong successes (significant long-term benefit, regularly use the ideas in daily life). If you want to try analysing the feedback yourself, you can see my anonymised summaries of all testimonials.
Two highlighted testimonials from the productive disagreements workshop that I’m particularly excited about:
In the interests of space, I give some highlights of the testimonials I am most excited about for the other workshops in Appendix B. I think reading things like this is most interesting for gauging the impact of teaching rationality in this format. But I also find that seeing how other people engage with techniques in practice can help me understand them more deeply myself, and help me see how to put them into practice
My prior was that short, one-off classes would not have any noticeable long-term impact, because they would be too short and easily forgotten, and not have a surrounding context of self-improvement to reinforce the ideas and get people to practice. Seeing this long-term feedback has strongly updated me towards thinking these kinds of workshops are valuable. The workshops didn’t have a significant long-term impact on most attendees, but had an impact on enough attendees that it seems extremely worth the total time investment to run and attend them.
I expect that certain kinds of people will get much more benefit from these workshops than others, and I had the useful filters of:
I expect these filters significantly increased expected benefit
I framed each workshop around a series of Trigger Action Pattern-style mental habits, eg “when I notice I am confused about what somebody is saying -> try paraphrasing it back to them”. I think this worked extremely well, and created a significant amount of the value of these workshops. A lot of the most successful feedback is people for whom these habits stuck, and are now regularly used. As far as I can tell, people didn’t put significant effort into deliberately retaining these habits, they just made intuitive sense and stuck. I am very pleasantly surprised that they stuck this easily. My rough model for this is that people remembered and used the habit shortly after the workshop, found it useful, and this reinforcement kept happening until the habit stuck.
One major weakness is that for the more practical workshops, people rarely put in meaningful effort to practice or retain the techniques after the workshop. Eg, in the systemisation workshop, I had participants design and implement a system in the workshop. Many found this useful, and had the system stick, but far fewer applied the ideas to design more systems afterwards. One guess for addressing this would be to have follow-up workshops entirely focused on applying and practising the techniques, as a form of group accountability. I’d be extremely interested in hearing any other ideas for addressing this problem!
In hindsight, I think most of the value of the workshops came from people doing exercises and practising the ideas, and less so from the content and theory. In future, I’d shift emphasis to spend less time talking, and spend more time on the exercises. Though I think there is still significant benefit to spending some time on theory, and trying to articulate the mindset behind why the techniques make sense and are useful.
I think teaching is an extremely important skill, and teaching skill and philosophy is responsible for a lot of the variance in how well people learn, so it’s something I try to think about a lot. In this section, I’m going to try to summarise my thoughts on teaching as relevant to applied rationality. If you’re interested, I go into a lot more detail on my thoughts on teaching generally in this blog post
One of the most important parts of my teaching philosophy is that learning is a process of information compression. We take in far more information than we retain. From a 90 minute workshop, most people will retain a few key points tops. This is important, because it means that I should be trying to choose those key points, and shaping the lesson around them. Fundamentally, the entire point of the workshop is to give context and reinforcement to those key points, everything else is irrelevant. Further, extracting those key points from a stream of information is significant intellectual labour. The student doesn’t know what is and is not important, and it takes effort to identify this, effort taken away from actually learning. Thus, as the teacher who does know what is and is not important, my role is to make it as easy as possible to identify these key points. Some strategies to achieve this:
Some ideas for teaching applied rationality specifically:
Overall, I think these workshops were a major success at actually conveying the techniques to the audience. I’ve gotten a lot of value from these ideas in my personal life, empirically they’re teachable, and I’d be excited about seeing these insights spreading and helping others to become more effective! I also found teaching these to be valuable personally, because it significantly clarified the ideas in my head. I’ve noticed myself using the ideas much more often in normal life as a result.
My lesson plans are here. I expect these to work best as a source to ad-lib from and adapt, rather than to follow perfectly, but I designed them to be detailed and thorough, so I hope they can save a significant amount of work! EA Stanford have run two of these workshops based on my lesson plans, and seemed to think it went well and was much lower effort to run than if writing workshops from scratch.
If you want to run one of these, to prepare, I would recommend making a copy of these notes, reading through in detail, and editing it to be in your voice. Eg, noticing the framings you dislike and changing them, replacing my examples with ones you relate with, noticing the points you don’t understand and cutting or thinking more about them. I think it’s important to understand what’s in the plan and to have it be something that makes sense to you, before trying to teach it to others. I generally err on the side of putting too much content into lesson plans, and cutting things when presenting (or overrunning, or both). I’d recommend cutting the parts of the lesson that don’t seem as interesting or exciting to you.
I’d be especially excited to see the productive disagreements workshop done in EA groups. To me, a key part of EA culture is having good epistemics - taking other people’s ideas seriously, trying to understand them, and being open to changing your mind. But I rarely see this norm explicitly set or taught, it seems more something that people are either already on board with, or pick up by osmosis. And I’m excited about seeing attempts to explicitly set culture.
Further thoughts on how to actually do this:
Overall, I think this was an excellent experiment! These ideas have become a powerful part of my mental toolkit, and I’ve been able to transfer them to others. My priors were against them being transferrable in this kind of context, but I’ve strongly updated in favour after doing long-term followups!
I’d be very excited to see other local groups trying to run these, and I hope my lesson plans can save some effort there. If you plan on organising these, I’d be very happy to chat and give any advice! Please feel totally free to reach out, and I’d be extremely interested in hearing how it goes if anybody does try them. My email is email@example.com
Thanks to all of the many people who took the time to give feedback on the draft! Especially Dan Keys, Luca Righetti, Nora Ammann and Nathan Young, who helped make this significantly better. And thanks to CFAR for introducing me to these ideas in the first place! A year on, I've gotten a ton of value from my workshop
The continuation of the content section for my workshops on planning, habits and systems. I summarise the motivation, key takeaways and structure of the workshop, and try to give Pedagogical-Content Knowledge - common misconceptions and specific insights for teaching the ideas well.
Some more examples of the testimonials I’m most excited about from the other 3 workshops:
I really enjoyed reading this, thanks for sharing.
I recently went through Mark Xu's "CFAR training regime" sequence and I'm looking for ways to strengthen what I learned there and share it with other people. Your post and lesson plans give me lot of ideas to think through and try to practice in the future.
Thanks for writing this up! This seems really useful for aspiring rationality organisers, will forward it to those I meet.