Thank you for writing this. Please set a reminder for yourself to follow up a year-or-so from now.
Sorry. I had to deliberately redact the details because, although it isn't particularly personal, it might cause some problems for me if the wrong people knew the specifics of what I'm looking into at this point in time. If and when that ceases to be the case I'll come back and replace it.
Understandable, this just seems like an odd way to redact the details. A footnote or something would have made it more obvious what you were doing.
Sorry if I confused you. Perhaps in the future I'll just say REDACTED or some such. I suppose I should also learn how to use footnotes in this editor.
First, I've become quite fond of the question "Does future me have a comparative advantage?" Especially for small items, if the answer is "No" (and it's no far more often than it's yes) then just do it right now.
Oh, this sounds like a very useful technique, thanks!
This is incredibly helpful. Thank you! This part stood out to me:
Similarly I completely did not understand the concepts of inside view vs. outside view at the workshop; and worse yet I don't think that I even realized that I didn't understand these. However now that I've read Thinking Fast and Slow, the lightbulb has gone on. Inside view is simply me deciding how likely I (or my team) is likely to accomplish something based on my judgement of the problem and our capabilities. Outside view is a statistical question about how people and teams like us have done when confronted with similar problems in the past. As long as there are similar teams and similar problems to compare with, the outside view is likely to be much more accurate.
This and the similar comment about having forgotten the Planning Kata makes for really useful feedback. Yours is not the only report we've recently collected like this, but I think this is the most detailed. We'll revise this and test some revisions. Thank you!
I wouldn't update a lot or revise too much based on this report. The simple fact is that there was so much packed into 4 days that there was just no way anyone could remember it all. I suspect different attendees understood, remembered, implemented, and forgot different subsets of the material.
I will note that it is extremely helpful to have the spiral bound notebook with detailed notes from the sessions. I've been skimming it every couple of weeks just to jog my memory, and give me ideas about what I should be working on. Usually I just toss these handouts after a conference or workshop, but this one's been really helpful.
I wouldn't update a lot or revise too much based on this report. The simple fact is that there was so much packed into 4 days that there was just no way anyone could remember it all. I suspect different attendees understood, remembered, implemented, and forgot different subsets of the material.
Noted, thanks. At the same time, the Planning Kata is a commonly forgotten one, and it's where we introduce the outside view. So it seems ripe for updating!
I will note that it is extremely helpful to have the spiral bound notebook with detailed notes from the sessions.
Good to know!
I forgot to mention one thing in reply to your top post, by the way: several references for Turbocharging Training should be in that booklet. The one part that the references don't particularly support is the addition of imagery as a means of creating intensity. It clearly works for many people, but the evidence is mostly anecdotal. Quite a few people walked away with the impression that the imagery was the point of Turbocharging (which it wasn't), so we've removed that and I now emphasize material that's more directly connected to the literature.
What are "COZE exercises"? Google isn't giving me anything.
Good article, its useful to see how these techniques are actually applied.
This is a CFAR-specific term, I think; it's short for "comfort zone expansion." Things like talking to strangers. The closest googleable analogue might be rejection therapy.
In bullfighting there is a term called querencia. The querencia is the spot in the ring to which the bull returns. Each bull has a different querencia, but as the bullfight continues, and the animal becomes more threatened, it returns more and more often to his spot.As he returns to his querencia, he becomes more predictable. And so, in the end, the matador is able to kill the bull because instead of trying something new, the bull returns to what is familiar. His comfort zone.
Winning at Arguments, I am already very, very good at when I want to be, which is rare these days. It took me many years too realize that even though I "won" almost every argument I cared about, winning the argument wasn't usually all that useful. Winning an argument is the wrong goal to have for almost any purpose, and rarely leads to the outcomes I desire.
If I recall correctly, the main point of this class was to redefine the meaning of "winning" to be something more like "achieving the optimal outcome for both parties"... i.e. you pause the argument as early as possible, consider the goals of everyone involved, and (ideally together but possibly with less cooperation) work to optimize for those goals. Your comment above sounds like it's still using the conventional definition of winning arguments, and it seems to me that you seem frustrated by that approach. Worth revisiting, perhaps?
Techniques for learning a new skill in any domain more efficiently and quickly. Finding the optimal intensity to work at to change it from something you have to do deliberately and consciously to something you can do automatically.
"Turbocharging Training" is just the proper name for the toolkit-for-learning-things that is taught in the CFAR "Turbocharging Training" class, so a full description would basically be that class.
I don't know how much of the class is original to CFAR but at least some of it is available in other sources. (E.g., if you're learning to play a piano piece, spend most of your time on the trouble spots, not on playing the entire piece from beginning to end. I've seen this advice in multiple non-CFAR places.)
This article was very helpful for me. Especially the part about dentists. It reminded me to call my dentist and make an appointment, which I've been procrastinating on for weeks. Anyone else in the same situation? Call now!
I am not sure how much evidence there is for Extreme Programming. But if it works, I wonder how much its rules can be translated to areas beyond programming. (Or maybe it already exists. Maybe "two minds work better than one" was already know for millenia, only for IT people it remains a surprising and controversial topic.) Could we somehow abstract the coding details away, and call it Extreme Doing?
The parts about planning (estimates, acceptance tests) seem universal enough. Sure, the tests will not be automatical, but writing them specifically and in advance could help a lot. Stand up meeting every morning? Trivial. What is a proper analogy for refactoring? Probably it means avoiding procrastination on strategic things; if I know that a small change in environment could make my future work more efficient, I should do that small change as soon as possible. Unit driven development? Before you start doing anything, write explicitly on paper which properties do you except from your result; later check the paper to confirm you did not forget anything. Pair Programming? Don't do things alone; you will have more fun. (Also the obvious problem: you need to find another fan of Extreme Doing working on the same project.) I am not sure about the continuous integration. It probably means that you should design your plans so that they will bring some partial benefits during the way to your goal, gradually increasing the benefits as you progress (as opposed to: first I must finish everything completely, and only then it starts being useful).
I am not sure how much evidence there is for Extreme Programming. But if it works, I wonder how much its rules can be translated to areas beyond programming. (Or maybe it already exists. Maybe "two minds work better than one" was already know for millenia, only for IT people it remains a surprising and controversial topic.) Could we somehow abstract the coding details away, and call it Extreme Doing?
It's controverisal, and application to other industries more so, but efforts to target some of the low-hanging fruit are underway.
The workshop introduced me to the concepts of System 1 and System 2. System 1 is the faster, reactive, intuitive mind that uses heuristics and experience to react quickly. System 2 is the slower, analytical, logical, mathematical mind.
Sounds like Korzybski. Is there the related concept of a "semantic pause" before taking action, to give System 2 the chance to weigh in on your response?
By the way, anyone know how long Korzybski recommended for a semantic pause?
The most immediately useful thing I learned was the Pomodoro Technique, as I've written about here before. In addition to that, there were a number of small items that I'm continuing to work on.
I've been having a bit of a problem with procrastination while working... the Pomodoro technique is working very well for me. Thanks!
A technique to subvert the planning fallacy, so you make specific plans that actually account for Murphy's law. (and avoid a few other common failure modes).
I was surprised that no one at the workshop seemed to have heard of Kanban or Scrum, much less practice it. Burndown charts and point-based estimation are a really interesting modification of the outside view by comparing your team to your team in the past, rather than to other teams.
I've worked with Scrum before. My general impression is that it's about one part useful to two parts extraneous formalism and fluff; a lot of what I interpret as fluff is probably designed to handle some failure mode I've never seen, but any given team is only going to exhibit a subset of possible failure modes, and the Scrum methodology doesn't encourage or give you the tools you'd need for that kind of tailoring. It does beat naive development styles, but that's mostly because naive development is incredibly inefficient: twenty percent technique, talent, and education and eighty percent not getting distracted by the Internet, to paraphrase a friend of mine.
The most useful change, when my team tried it, seemed to be offloading most coordination burden into short, regularly scheduled informal meetings at less productive times of day rather than indigestible semi-random clots scattered throughout the week. That helped quite a bit, although it also created scheduling problems of its own.
Recently Leah Libresco asked attendees at the January CFAR Workshop, "What habits have people installed after workshops?" and that got me thinking that now was a good time to write up and review what I learned (or learned and already forgot). I thought that might be of some interest to folks here, and this is what follows.
The most immediately useful thing I learned was the Pomodoro Technique, as I've written about here before. In addition to that, there were a number of small items that I'm continuing to work on.
First, I've become quite fond of the question "Does future me have a comparative advantage?" Especially for small items, if the answer is "No" (and it's no far more often than it's yes) then just do it right now. The more trivial the task, the more useful it is. For instance, today I asked myself that while standing in the bedroom wondering whether to take 30 seconds to move my ExOfficio Bugproof socks from the dresser to the correct box in the closet. (Answer from a few minutes ago: if I don't take my dog for a walk right now, he's going to pee all over the floor. Future me does have a comparative advantage of not having to clean up pee on the floor. The socks can wait.)
I've begun to notice my confusion and call it to conscious attention more often, though I suspect I learned this first from HpMOR and the sequences before the workshop. Example: when Leonard Susskind states that conservation of information is a fundamental principle of quantum mechanics, I notice that I am confused because A) I have never heard of any such fundamental law of physics as information conservation B) Every definition of information I have ever heard indicates that information most certainly can be destroyed. So just what the heck is he talking about anyway? I am now making a conscious effort to research this topic rather than letting it slide by.
The workshop introduced me to the concepts of System 1 and System 2. System 1 is the faster, reactive, intuitive mind that uses heuristics and experience to react quickly. System 2 is the slower, analytical, logical, mathematical mind. I didn't immediately grok this or see how to apply it. However the workshop did convince me to read Daniel Kahneman's Thinking Fast and Slow, and I'm beginning to follow this. It could be useful going forward. I particularly like the examples given at the end of each chapter.
Similarly I completely did not understand the concepts of inside view vs. outside view at the workshop; and worse yet I don't think that I even realized that I didn't understand these. However now that I've read Thinking Fast and Slow, the lightbulb has gone on. Inside view is simply me deciding how likely I (or my team) is likely to accomplish something based on my judgement of the problem and our capabilities. Outside view is a statistical question about how people and teams like us have done when confronted with similar problems in the past. As long as there are similar teams and similar problems to compare with, the outside view is likely to be much more accurate.
During conversation, Julia Galef and I came up with the idea of *********. It turned out it already exists, and I'm planning to start attending these events locally soon. I've also joined my local LessWrong meetup group.
Stare into Ugh fields. Difficult conversations are an Ugh field for me. Recognizing this and bringing it to conscious attention has made it somewhat easier to manage these conversations. Example: when I went to the workshop I had been putting off contacting my dentist for months, not because of the usual reasons people don't like going to the dentist, but simply because I was uncomfortable telling her that the second (and third) opinion I had gotten on a dental issue disagreed with her about the proper course of treatment. Post-workshop, I finally called her (though it still took me two more weeks to do this. Clearly I have a lot of work left to do here.)
Consider whether the sources of my information may be correlated and by how much. I.e. Evaluating Advice. For instance, if two dentists who share an office give me the same advice, even assuming no prior disposition to agree with each other simply out of friendship, how likely is it that they share the same background and information that dentists in a different office do not?
COZE (Comfort Zone Expansion) exercises have pushed me to talk more to "strangers" and be intentionally more extroverted. On a recent trip to Latin America, I even made an effort to use what little Spanish I possess. I've had some small success, though this has led to no obvious major improvements in my life yet.
Thought experiments conducted at the workshop were very helpful in untangling some of my goals and plans. Going forward though this hasn't made a huge difference in my day-to-day life. That is, it hasn't led me to seek different paths than what I'm on right now.
Going over my notes now, there was a lot of material; some of it potentially useful, that has fallen by the wayside; and may be worth a second look. This includes:
Value of Information calculations seem too meta and too wishy-washy to be of much use. They attempt to put quantitative numbers based on information that's far too imprecise to allow even order of magnitude accuracy. I'm better off just keeping things I need to consider in my GTD system, and periodically reviewing it.
Similarly opportunities for Bayesian Strength of Evidence calculations, just don't seem to come up in my day-to-day life. The question for me is more commonly "Given that the situation is what it is, what actions should I take to accomplish my goals?" The outside view is useful for this. Figuring out why the situation is what it is rarely seems to be especially helpful.
Turbocharging Training may be helpful but the evidence seems to me to be lacking. I'd like to see some strong proof that this works in particular areas; e.g. foreign languages, sports, or mathematics. Furthermore, it's not clear that it's applicable to anything I'm working on learning at this time. It seems very System 1 focused, and not especially helpful with the sort of fundamentally System 2 tasks I take on.
I have begun to declare "Victory!" at the end of a meeting/discussion. it's a bit of fun, but has limited effect. Beyond that I don't seem to reward myself for noticing things, or as a means of installing habits.
Getting Things Done (GTD), Remember the Milk, BeeMinder, Anki, Cultivating Curiosity, Overcoming Procrastination, and Winning at Arguments.
GTD I didn't learn because I've used it for years now or at least the parts of it that really work for me (lists and calendars mostly, and to a lesser extent filing).
Remember the Milk because my employer's security policy prohibits us from using it, and too much of my life happens at my day job to make maintaining two separate systems worthwhile.
BeeMinder and Anki because I just don't have anything that seems it could benefit from being stored in those systems right now. All of these might be more beneficial to someone in different circumstances.
Cultivating Curiosity because I am already a very naturally curious person, and have been for as long as I can remember. I don't need help with this. Indeed if anything I need to tamp down on this tendency and focus more on accomplishing things rather than merely learning them.
Similarly, Overcoming Procrastination didn't help a lot because I don't have a big procrastination problem, at least not compared to what I had when I was younger. Of course, I do say that in full knowledge that right this minute writing this article is a form of structured procrastination to avoid doing my taxes. :-)
Winning at Arguments, I am already very, very good at when I want to be, which is rare these days. It took me many years too realize that even though I "won" almost every argument I cared about, winning the argument wasn't usually all that useful. Winning an argument is the wrong goal to have for almost any purpose, and rarely leads to the outcomes I desire.
Polyphasic sleep: I'm going to let the younger, more pioneering attendees experiment with this one. Even if it does work (which seems far from obvious) I don't see how one could integrate it into a conventional day job and family.
At breakfast one morning, a fellow attendee (Hunter?) suggested putting unsalted butter in my coffee to add more fat to my diet. It's not as crazy as it sounds. After all butter is little more than clarified cream, which I do like in my coffee. I tried this once and I still prefer cream, but I may give it another shot.
Finally, I've referred two workshop attendees to my employer as potential hires. If anyone else from the workshop is looking for a job, especially in tech, sales, or legal, drop me a line privately. For that matter if any Less Wronger is looking for a job, drop me a line privately. We have hundreds of open positions in major cities around the world. Quite a few LessWrongers already work there, and there's room for many more.
There were a few techniques that were conspicuous by their absence. In particular I think the CFAR/LessWrong and Agile/XP communities have a lot to teach each other. I was surprised that no one at the workshop seemed to have heard of Kanban or Scrum, much less practice it. Burndown charts and point-based estimation are a really interesting modification of the outside view by comparing your team to your team in the past, rather than to other teams.
Pairing is also a useful technique beyond programming as at least Eliezer (not present at the workshop) has discovered. Pairing is an incredibly effective way to overcome akrasia and procrastination.
In reverse, I am considering what the craft of software development has to learn from CFAR style rationality, more specifically epistemic rationality. I have begun to notice my confusion during conversations with users, product managers, and tech leads and call it to conscious attention. I less frequently let unclear specs and goals pass without comment. Rather, I ask for examples and drill down into them until I feel my confusion has been conquered.
So far these techniques seem very useful in analysis and requirements gathering. I've found them less obviously useful (though certainly not harmful in any way) during coding, debugging, and testing. In these stages there's simply too much to be confused by to address it all, and whatever I'm confused by that's relevant to the task at hand rapidly calls itself to my attention. For instance, when a bug shows up in a production system, the very first and natural question to ask is "How the hell did the system do that?!" On the other hand, the planning kata may be very helpful with the early stages of system design, though I haven't yet had an opportunity to try that out.
Overall, I found the workshop to be a worthwhile experience, if an expensive one; and I recommend it to you if you have the opportunity and resources to attend. There are a lot of practical techniques to be learned, and you only need one or two of them to pay off to cover the cost and time. Even if the primary value is simply introducing you to books and techniques you explore further after the workshop such as Getting Things Done or Thinking Fast and Slow, that may be enough. Most knowledge workers are operating far below the level of which we're capable, and expanding our effectiveness can pay for itself.
Before attending, it is worth asking yourself whether there's an opportunity to learn this material at lower cost. For instance, did I really need to spend $3900 and 4 days to learn about Pomodoro? Apparently so, since I'd heard about Pomodoro for years and paid no attention to it until January. On the other hand, a $20 book I read on the subway was fully sufficient for me to learn and implement Getting Things Done. You'll have to judge this one for yourself.