I am constantly in analysis paralysis. I personally waste way too much time trying to prematurely optimize how I am going to learn something before actually learning any piece of the issue. To remedy this, I try to tell myself that I will naturally optimize the learning process as I progress. I think this was nicely explained by Andrej Karpathy in a podcast.
Recently, I have been thinking about this a lot as I am on full-time sabbatical, learning technical alignment research. There is so much to do and a general feeling of ignorance in any direction, making it ripe for constant optimization. I would love to have an answer, but at this point I try to operate mostly on the faith that I will optimize and the more I force premature optimization, the more time I waste. At some point, I have to just Do The Work.
I like where this ends. For lots of things, the (potentially frustrating) reality is that we must simply have the faith to do the work we know we should do and trust that it will have the intended effect. Trying to speed up early often works against the value of doing the work, and costs more in the end than simply doing the work would have.
faster is often the wrong dimension for me to focus on. Though good optimization does lead to speed, it often feels like a result rather than a direct optimization target. I think something closer to how could I have learned that more easily, more completely. What was extraneous in retrospect? Were there any signs that I could have noticed earlier? What could I have included earlier that was obviously helpful?
One dynamic these questions reveal is often subtle bike-shedding: focusing on those areas that gave quick hits of sense of progress rather than what moves the needle on the original goal the most. A shorthand I have for this is 'decision leverage', as in how will what I am doing currently connect to something I do differently in the future?
More generally, the four pedagogical interventions with the largest effect size AFAIK are
deliberate practice
elaboration of context (connection to other areas of knowledge)
frequent low stakes quizzing
teaching the material to others
I'd just note, that you should be cautious of people "answering" this question in hindsight.
In both of the two subjects that I feel most professionally confident in and have had the chance to teach (maths and computer science) you'll see people sharing a common refrain. "If only I'd learnt {Complicated method/language/Mental Model} first, I'd have saved myself so much time."
The most common examples I've seen of this are people who are convinced that teaching kids pointer juggling is gonna give them a stronger foundation for CS, or the cult of "Linear Algebra Done Right" (a book that I love, but that isn't a good introduction to the field imo).
"Lies to children" exist for a reason, and while some might be skippable, many form useful intellectual scaffolds.
This is a question, but also a linkpost from my (new) Substack.
“How could I have thought that faster?” is great advice I am embarrassed I did not think of myself. A subset of this, I would say, is not thinking original thoughts, but rather the ability to effectively learn new things. What prompted this is my continual inability to find all the papers relevant to a question I am researching in a timely manner.
I have been reading about biological learning rules for four years, albeit not continuously, yet still manage to find papers that are both old and have all the keywords that I routinely search. I wonder how I missed them, if I found them before and ignored them for some reason. But the big fear isn’t about any paper in particular, it’s that I won’t be able to fill in huge knowledge gaps when the answer is out there.
There is an especially humorous, or disappointing, example of this happening to someone else: that being a biologist rediscovering calculus in 1994.
I think that this poses a real problem to academics in general and will only worsen as more and more research is published that could be applicable to your problem.
I also think it is worse for autodidacts or people otherwise learning alone, having structure and people around you lets you leverage existing knowledge much easier.
Maybe this is just me being bad at using search engines/the new wave of LLMs but it is a big frustration. I don’t think tools like semantic scholar fully solve the problem either, at least not for me.
I think this generalizes past finding papers to read. It is more of a failure to know what you should be spending your time doing during the skill/information acquisition stage of a project. When you start a project, you usually aren’t sure how you are going to solve all the problems that pop up along the way. If you don’t already have a huge bag of problem solving techniques that are applicable to that domain, it is hard to know where you should be allocating your time to learn what will be useful and not a bunch of extraneous knowledge vs. actually getting to work.
I guess what I am really asking is how best to solve these instances of the explore vs. exploit problem. How do you know when you’re informed? If you want to make the declaration that a topic is not well researched academically, how do you know you have exhausted the literature enough?
I have been burned on this so many times I have become very enamored by the idea of going all the way back to basics. This is enjoyable to a certain extent and almost ensures I don’t miss anything but it is probably not optimal in many ways. LW has discussed the use of reading textbooks as they are an underrated way of gaining knowledge. I have somewhat successfully rolled this into my belief system but it still doesn’t feel like making progress. The questions I have become interested in are now in neuroscience but I am going through a math textbook. I think it will be useful but is it optimal? Am I just going to have to eat a year of study before I can start work on things I actually care about?