You Can Now Embed Flashcard Quizzes in Your LessWrong posts!

[-]Gordon Seidoh Worley4y50

Registering my continued dislike for dedicating effort on this feature (I voiced similar complaints when this was originally floated). My reasons:

retaining knowledge via spaced repetition is a poor fit for most of the posts on LW
the general advice is that using cards written by others is much less useful than cards you write yourself
nudges LW towards encouraging posts that are more amenable to space repetition, which I expect will marginally decrease other kinds of content
- implicit in this is an argument that LW would be marginally worse if it spent more time on teaching things that you can learn by remembering them and less time on other things, like working towards reducing confusion about things we are currently confused about

[-]jimrandomh4y140

When I originally floated the idea on Facebook, I was considering implementing it as an LW feature directly, which would've been a lot of effort. Then it turned out ThoughtSaver was already building it, with cross-site integration in mind. The actual implementation on LessWrong works the same way as the Metaculus and Elicit integrations we already had, and took less than an hour on our end. (Much more than an hour of effort on ThoughtSaver's end, obvciously, but amortized across more applications than just LessWrong integration, and that part of the effort didn't come from the LessWrong team.)

[-]spencerg4y120

Fair enough, though I disagree with these points. For one thing, this feature already works, so I don't expect it will take any significant amount of time from the LessWrong dev team.

But to respond to your more specific points, from my perspective:

(1) there is a common misconception that flashcards and/or spaced repetition is mainly useful for facts and definitions. It is also very useful for concepts ("a generative adversarial network is..."), takeaways (e.g., "the most important three points this post makes are..."), connections between ideas ("X and Y are thought to be different but they are connected through Z..."), strategies ("Here's a process you can use..."), and so on. I certainly don't think all LessWrong posts are suitable for this, but I think you're underestimating how many are. My guess is it is a good fit for 5%-20% of posts but that's a super rough estimate. I personally use flashcards for complex concepts all the time, including plenty I've created for myself from LessWrong and Slatestar Codex posts.

(2) indeed it is a standard belief that if you pick up someone's random flashcard deck that they wrote for themself you are unlikely to get much out of it (I agree with this too). That is different from a deck written specifically by the author for an essay (almost nobody has experience with this, other than through the experiments I link to in the bottom of the post) so the "I can't pick up my friend's deck" argument doesn't have much weight in this case in my view.

(3) I would argue that even posts that e.g., "work towards reducing confusion about things we are currently confused about" have takeaways that are worth remembering (e..g, "what should you think about differently having read the post?", "what was the approach of reducing the confusion?", "what is a useful analogy for thinking about this topic?", etc.)

[-]Gordon Seidoh Worley4y70

These are all quite reasonable, and I'm pretty open to the idea that I'm mistaken and anchoring too much on the fact that I didn't find flash cards for spaced repetition useful, which might ultimately bias how heavily I weight things or assess the likelihood that flashcards would be helpful.

[-]ChristianKl4y60

The process of making a piece of knowledge suitable for space repetition is a lot about making the piece of knowledge as clear as possible.

When it comes to my own writing, I don't think it discourages me to write about ideas I believe are worth writing about that this new feature exists. I might find it interesting to experiment with a new way of writing posts and use the feature, but I don't believe that this is zero-sum.

Asking "Is this likely to be useful?" is the wrong question. I think it's better to ask: "Is this an experiment that has the potential to produce something useful for the art of rationality?"

Trying out new ways to represent knowledge about the art of rationality has the potential to lead to progress and is thus worth doing. We shouldn't try to focus on one approach of gathering knowledge about rationality but try many.

[-]MikkW4y60

The general advice is that using cards written by others is much less useful than cards you write yourself

As someone who has used SRS on a regular basis for multiple years, I disagree with this. Sure, there are times when my best option is to hand-craft cards for myself, but this is usually a function of there not already existing high quality cards that test me on the things I want to learn. Creating cards is useful for allowing me to focus on exactly the facts that I want to learn; the other times where this is true is when I have a specific model and vocabulary for understanding a field that isn't standard; there, creating cards allows me to target the specific vocabulary and ontology that is unique to me.

However, often times I want to learn something that doesn't meet these criteria: things that I can easily think about using a standard ontology and vocabulary, and are common enough subjects that someone has already taken the effort to make a sufficiently high-quality deck. In this case, making my own cards is just a waste of my time (and the time that is wasted isn't negligible; I've spent up to an hour on multiple occasions creating cards for domains that could just as easily been downloaded if I knew of a high quality deck that already existed). I'm using premade decks for learning pronunciation and meanings of Japanese Kanji and for mapping flags to the countries they represent. There are many subjects where I have cards I made myself that I would be just as well off using equally high quality cards made by another person, and others would be well-served using the same cards I made: learning the pronunciation and meanings of Russian words, mapping coordinates to the cities they locate, memorizing poetry, mapping astronomical symbols to the bodies they represent. Even in the case of mathematical definitions and associations, while the cards I use are tailored to me, similar cards could certainly be made that would be suited to a much wider audience, that would be nearly just as useful as the cards I actually use.

I'd estimate about 2/3rds of the cards in my Anki deck are too specific to me to have been mass produced, but the other 1/3rd would be very well justified in having been made with the intention of being able to be used by others, and that ratio could be even higher (maybe up to 2/3rds mass produceable) without causing problems.

[-]Kaj_Sotala4y50

the general advice is that using cards written by others is much less useful than cards you write yourself

Seems like using cards by others is still better than not using any cards at all?

[-]ChristianKl4y20

There's an opportunity cost. If you spend a lot of time memorizing cards that don't really fit into your map of the world that costs a lot of time.

[-]David Hartsough4y50

Can't wait for this book summary! It's a very rad read for rationalists indeed!

But in all seriousness, I'm excited to see what neat new posts will utilize this, and I'm curious to know how this might impact writers who are trying to think of ways to provide their readers with a great takeaway. ("Something to remember me by" haha)

For me, I'm already thinking about how to write the flashcards for a post in such a way that each key thought might be somewhat independent on its own yet containing enough context to relate it back to the collection of thoughts that map/model the ideas behind a single post/concept. If I were to encounter one flashcard from the post's deck in a year from now, would I then easily recollect the entire post and its most important thoughts? I like the framework of thinking of each post as having one correlated deck of flashcards, and each key idea within that post correlating to one flashcard. The flashcard deck can represent a graph/network of thoughts that you want to remember "forever", and hopefully if you encounter any "node" (i.e. a flashcard) from that graph (i.e. the deck / the core concept of the post), then you can reinforce the connections between those ideas and strengthen your memory and understanding of the concept.

(On a tangent, this kinda makes me wonder: "What decks of flashcards would I have if I had taken this approach to learning since my childhood? What if my schools had left me with flashcard decks like this to take with me for the rest of my life? Which bits of information are most essential to hold on to forever?")

[-]spencerg4y40

A big shout-out to David for making this feature happen :)

[-]Chris_Leong4y40

This is a cool feature, but I'd honestly be surprised if it gets much use.

[-]eigen4y30

This is amazing. Incredible execution, it does not go unnoticed!

[-]spencerg4y10

Thanks :) Glad you think so!

[-]riceissa4y30

I'm curious why you decided to make an entirely new platform (Thought Saver) rather than using Andy's Orbit platform.

[-]spencerg4y50

A few thoughts on this:

• I don't think Orbit existed when we started working on Thought Saver (or if it did, we didn't know about it). When Quantum Country came out I wasn't aware (not sure if anyone was aware?) at the time that it was part of a larger project (maybe it wasn't yet at that point?)

• Thought Saver has a different focus than Orbit (e.g., Thought Saver is not focussed on embedded essays - it just seems that way because the LessWrong team asked if we could add that feature since they wanted to try embedded flashcard quizzes). Thought Saver is more fundamentally focussed on being a tool for accelerating your own learning.

• Also, as far as I'm aware Orbit can't be used in a self-serve fashion right now (?)

• I'll add that Andy is awesome, I deeply respect his work, and I consider him the world expert on the topic of writing essays with memory elements (and on other topics related to spaced repetition as well). I'm excited to see what he does with Orbit!

[-]andymatuschak4y60

Thanks, Spencer—I'm excited to see what you learn from this. :)

FWIW, Orbit can indeed be used self-serve, but its OEmbed support (for embedding into user-generated content editors like on here, Medium, GitHub, etc) is not yet available; you must be able to embed HTML blocks. I've not yet advertised this very broadly because I've been looking to cultivate closer relationships with authors in contexts where the medium might be especially useful. Like many commenters here, I fear the existing embedding interaction is probably a net negative in many contexts.

[-]philh4y20

Is this available for automatic crossposting? I guess thought saver will have some way to embed in arbitrary html (iframes? I haven't done this kind of thing much) but will that work when crossposted?

I don't know if I'll use this much, so don't put effort into making it work on my account, just wondering.

[-]spencerg4y10

Hi, I'm not sure if this works automatically with cross-posting - depends what you mean exactly. You'd have to set it up on each website you want the cards to appear on.

[-]Yoav Ravid4y*20

wesome! I thought about such a feature before but didn't expect to suddenly see it added. I'm really curious to see how it will be used. In the meantime, since this post doesn't have flashcards, I get to try making it some flashcards myself :)

~~uh... Well i tried, but for some reason it created a new deck (with the same name) for each card I created, not really sure why it did that or how to fix it.~~

Problem was fixed, here are the flashcards:

[-]spencerg4y10

Hi Yoav! It should be putting them all into the same deck. If you click the search bar it should show all your decks. Can you click feedback in the upper right and tell us what you see there? We've never encountered an issue like the one you're describing. We'd like to help you get to the bottom of it.

Thanks!

[-]Yoav Ravid4y20

When I click the search bar it shows the starting deck I added (How to make the most of thought saver) and 6 decks with the same name (which is the number of cards I created for it)