Followup to: Spaced Repetition Database for Mysterious Answers to Mysterious Questions

I've updated my Anki database for the Less Wrong Sequences to include cards from A Human's Guide to Words. I've been trying to put less information on each card, and I relied on cloze deletion more for the newer ones.  Feedback is much appreciated. You can download them by opening up Anki, going to Download > Shared Deck and searching for Less Wrong Sequences.

I probably erred on the side of making way too many cards, but it seemed really important to me to internalize this stuff, since I think it has quite a lot of practical value. I can tell learning this deck has improved the quality of my thinking and my conversations with people because I'm better at noticing when I'm making one of the 37 mistakes and changing my course. I hope other people find it useful too!

New Comment
37 comments, sorted by Click to highlight new comments since:

I made a deck of the list of cognitive biases and list of fallacies from wikipedia.

Okay, holy crap Divia. That is a lot of cards.

As an Anki user (yes I switched!), I would have the cards using a model where the source post is in a separate field, perhaps with the url as another field. I guess if we're trying to stick to Q/A for compatability with other SRS systems, that's not a good idea, and what I'm suggesting is a horrific amount of work if you were to do it by hand, because you'd have to redo all the cards. So, maybe these are goals for the long term, in case SRS learning really increases in popularity, and Anki decks become a good vehicle for delivery in of themselves.

Obviously I haven't actually worked my way through the new additions (or even through most of the existing ones) but I think the cloze deletions are a good change. Overall I think your later cards reflect your increasing wisdom for structuring and formatting the cards.

Thanks and keep up the great work!

Having the source post in a separate field seems like a really good idea, and it never crossed my mind before, so thanks! It would be enough work that I'd probably write some script to do it instead of doing it by hand, but it might be worth it to me to do so at some point.


You can do it by exporting the deck to csv, then using a emacs macro to automate the transition.


Thank you so much for posting this! I use anki a lot, and your Mysterious Questions deck has been a great help =]

Care to share your experiences with Anki? I'm just starting using it, and I have several qualms and questions. First of all, what is the proper way to select a sub-deck with hard cards and drill through them repeatedly? Second, if you are learning languages, what is your approach to grammatical notes and multiple word forms, and, generally, what do you do when you need to have more that just two pieces of information linked, as it is often the case with irregular verbs? Hope you don't mind my asking.

  • Normal flashcards should be all equally difficult: as easy as possible. The idea is to break everything down into atomic facts; this makes it so you can't short-circuit a difficult card by just memorizing the answer; by memorizing all the parts, you still have the whole.

  • If you really want to drill one sub-deck, you can choose "cram mode" , and select the tag of the cards you want to review.

  • I don't use anki for languages, but to learn conjugations of verbs, I would have many example sentences with a "... " where the verb should go. You could ask on #anki or the google group. Here's a good article on how to make effective flashcards from the inventor of the spaced repetition algorithm, Piotr Wozniak.

  • Unconventional decks like having anki cards for a whole piano piece or problem in a textbook might work, but I haven't tried them... yet. I'll be experimenting with those this coming semester.

Unconventional decks like having anki cards for a whole piano piece or problem in a textbook might work

I have used Anki for learning bass guitar parts to songs, and I found this method: break a piece up into individual riffs or themes, make flashcards with the "[name of song] [riff or theme sheet music / tablature]". Add in flashcards that use cloze deletion on a list of how the riffs progress (intro -> verse -> ... -> verse -> chorus -> coda, for example, deleting chorus) and you have 10-20 cards, depending on complexity. I also threw in transitional licks appropriate to the song to bump the count up.

When testing yourself with the deck, have your instrument at hand. On riff cards, practice the riff for five minutes, then hit 'again'. Answer the other cards as normal - I originally planned to play the riff that was deleted, but I found it wasn't really necessary. I suggest responding 'again' to each riff card until you can play the riff first try.

The bonus of this method is it works out to ~1 hour of practice of your instrument. I found huge improvements because it ensured 80% of my hour was spent actually learning and improving rather than practicing well-known or ingrained patterns. My recall of songs from name was significantly higher too.


Thank you! I was planning on setting up a system for piano and guitar and I wasn't really sure what would work. This sounds great =]

Note that it might work better if you simply play the riff or theme once; my system has this weird time dynamic where some cards can take 5 minutes and be repeated three or four times, where others take 5 seconds and are dismissed first time around. This may not play nice with the spaced repetition algorithms.

Normal flashcards should be all equally difficult: as easy as possible. The idea is to break everything down into atomic facts; this makes it so you can't short-circuit a difficult card by just memorizing the answer; by memorizing all the parts, you still have the whole.

In my experience, I've found this to not be as true as it seems. I originally had many of my cards as atomic as possible, based on what Piotr Wozniak suggests, but while I had each individual part memorized, they all seemed to hang loosely connected, without a vivid thread holding it all together. Two clear examples for me are memorizing chess moves and poetry: Originally I'd memorize poems line by line, with each line shown in its surrounding context to prompt me, and chess games move-by-move, prompted by the state of the board at that moment.

I later experimented with more coarse chunks, where a single card would represent an entire stanza of a poem, or 10 ply of moves from a high-tier chess game. The cards took longer per interaction, but only a little longer (since I didn't have to switch contexts, increasing short-term flow), and the relativly fewer number of cards needed to represent the information more than made up for the longer reviews. But the most important benefit was that things suddenly became much clearer for me. Instead of being vaguely aware of the contours of the poems, I would find myself reciting them wholesale in the shower, unprompted (even fairly long poems like Tennyson's Ulysses, which clocks in at 70 lines - which, by the way, I shudder at the thought of trying to memorize line-by-line, like I have some shorter poems). Suddenly, I would see an entire series of chess moves as being deeply interconnected, in a way I never noticed trying to memorize those same moves one at a time.

The main benefit I thought atomization would provide, namely that each element would be readily available for combination with other, seemingly unrelated ideas, I don't really notice much loss of. While each element is richly situated in an existing context, I can easily pluck that idea out of that context to join it with ideas from another, but the richness of the existing context makes it much easier to get a handle on that idea in the first place

Some more thoughts: there's one card in my notes that asks:

"According to Ray Dalio, when you have a problem, you should ask yourself 6 questions: What went wrong? Have you made a mistake like this before? What was the immediate cause of the problem? What was the root cause of the problem? What can you do to correct the problem in the short term? What can you do to prevent problems like these in the long term?"

All 6 questions are hidden on the same card, so I have to provide all 6 in order to mark the card correct. Following the principle of atomization, I might have thought to create 6 different cards, one for each question to ask, but my experience with such atomized lists tends to be that each card is much more slippery, and the different items would all blend together in my mind (creating problems both during review and when the opportunity for application arises), whereas when the list is presented as a cohesive whole, it's much easier to remember each part as a part of the whole, and makes each part contrast better with the other items

I've tried to learn Esperanto and French using Anki. I'd recommend that you don't actually explicitly learn the grammar of your target language. For fluency, you need to be able to use correct grammar without conscious thought. Using grammar SRS cards, eg. 'conjugate this verb', will enable you to know correct grammar, but not at the intuitive, subconscious level you need for real fluency.

The best way around this, it seems, is to train RECOGNITION of the meanings of many example sentences. This can be done two ways.

Firstly, through lots of exposure to media in the target language, eg. audiobooks. You don't need to understand what's being said, so long as you familiarize youself with the pronunciation, tone and stress patterns of the language. You will gradually start to understand what's being said, both from your SRS (see below) and just through osmosis, the way small children learn their home languages.

Secondly, by training understanding of many sentences in the language through SRS. You should not try to translate English sentences to sentences in your target language, you should only try to understand whatever the sentence in the target language means. In an SRS, you can add the sentence (in your target language) to the question field. Leave the answer field blank. If you fail to understand the sentence, look up all the words, idioms, etc. that you don't understand and add them to the answer field.

More on this technique:

Using this ought to make me comfortable enough with Anki to actually begin using it. I have for a while suspected that while rationality is seeping into my brain, I am only using it on LessWrong, and not remembering it anywhere else. Spaced repetition seems like the silver bullet for that.


Wow, that must have been a massive amount of work, thanks!

I don't think it's the kind of knowledge I'd want to rote-learn through spaced repetition, but I think it 'still interesting in the way reading the monthly rationality quotes on LW is.

Has anyone created a deck for An Intuitive Explanation of Bayes' Theorem?

If not, would there be a lot of interest in a deck for An Intuitive Explanation of Bayes' Theorem? I feel like spaced repetition and continuous correction of my understanding is the only way I will actually becomes more Bayesian, rather than merely thinking it would be really cool to be more Bayesian.

I need to update my model of learning for this.

I've thought about doing it myself, but it seems like a challenging topic.

I just started using this and it it's pretty great. Thanks for all your hard work!


Has anyone gotten the (pricey - $25) Anki iphone app? Does the app still make it easy to look up the Less Wrong deck? Alternately, has someone tried importing the deck into some (cheaper) SRS app?


I have the Anki iphone app. Considering the utility and convenience it provides, the price is negligible. For comparison, at a private college, tuition/# of classes ~= $200 / class, so as I use anki for schoolwork, it easily pays for itself.

If you do any sort of utility calculation for products you use, a lot of times convenience will trump price by orders of magnitude. This is one of those cases.

Anki decks can be exported into the lowest-common-denominator format among SRS apps: tab-delimited text files. They're supported by pretty much every app worth using, and if they aren't, easy to transform them into something the app does understand.

From the discussions on the Mnemosyne-proj-user mailing list which I didn't pay any attention to, there are apparently a number of free SRS apps for the iPhone.

(What isn't usually all that portable is the markup for formatting, images, audio, etc. But I don't think any of these cards are likely to involve much of that.)

Anki decks can be exported into . . . tab-delimited text files.

If someone were to post a link to Divia's Guide to Words as tab-delimited text, I would look through it.

I know a full featured app is much better, but Anki Online is completely free, and accessible via most browsers--although it requires an internet connection. Any deck you sync from the desktop (or other) version of Anki should be available via AnkiOnline.

Wow. I have a free Anki app on my Android phone.

Thanks, this has been very helpful, I've previously used the wiki and personal notes, but this contains a decent summary.

Divia, this is brilliant.


One only slightly related question: from

While you use the software, detailed statistics can be kept on your learning process. If you want, these logs can be uploaded in a transparent and anonymous way to a central server for analysis.

This data will be valuable to study the behaviour of our memory over a very long time period. As an additional benefit, the results will be used to improve the scheduling algorithms behind the software even further.

Does anybody here know how efficient the current scheduling is already, or how much differences there are between different people/topics/whatever?

Does anybody here know how efficient the current scheduling is already,

Mnemosyne uses one of the older Supermemo SRS algorithms; Peter (the Mnemosyne dev) has said in the past that the more recent & complicated Supermemo algorithms offer little advantage.

or how much differences there are between different people/topics/whatever?

I dunno. The relevant statistic is probably 'easiness'. IIRC, there is in fact a deck-wide easiness factor that is slowly adjusted to deal with users who systematically make very slowly or very quickly learned cards. You could try looking at the torrent of Mnemosyne statistics and see how widely that statistic varies from person to person?

I am not quite following this. I just downloaded AnkiDroid. When I open the menu and choose "get shared deck", I get a huge list of decks. Is there another way to find the Less Wrong Sequence than going through this long, long list?

I don't have AnkiDroid, so I'm not sure if there's a way to search the shared decks within it, but if not I bet it would work to get the desktop version, sync it with an online account, and then sync AnkiDroid with the online account.

Confirmed. (Specific data point was the NVC deck, but I am confident that it generalizes.)

are the LW sequence decks available for Mnemosyne?

[Edit: Divia posted this one above, while I was composing this comment: ]

Unless Divia has something better, here's a rough export to Mnemosyne:

LW Sequences .mem Deck: lw-sequences.mem

LW Sequences cards in a tab-delimited file: lw-sequences.txt

Couldn't figure out how to preserve the tags. AFAICT, Mnemosyne doesn't support importing them at present.

(Psst, Zach, maybe I should've told you this earlier, but I switched over to Anki! It was a little bit painful, since I had to abandon learning info on 600 or so cards, but Anki is just that good that I'm not sorry at all. I encourage you to continue to use what you're comfortable with and will actually learn with, but it's worth watching the Anki vids! Among other things, Anki natively supports syncing across computers, and it's possible to access the decks you've synced online via a web browser... Just sayin'.)

LW Sequences .mem Deck: lw-sequences.mem

Mnemosyne's XML is strongly suggested, I think. For example, .mem (Python pickle format) will be going away in Mnemosyne 2.0 in favor of an SQLite database.

Couldn't figure out how to preserve the tags. AFAICT, Mnemosyne doesn't support importing them at present.

Treat tags as categories? Unless you really do have cards with multiple tags. Or wait for Mnemosyne 2.0, which loosens the categories into tags.

I just downloaded Mnemosyne yesterday, so its not too late to test both softwares.