Epistemic status: Exploratory
I previously wrote an essay about Scaffolding Skills. The short explanation is that some skills aren’t the thing you’re actually trying to get good at, but they help you learn those things. Literal scaffolds are the temporary thin platforms you see on the outside of construction projects, which give builders a place to stand when working on a more permanent building.
The two sentence summary of this essay is this: In martial arts dojos your sensei can watch how you throw a punch, and give you feedback based on what mistakes you're making. I believe the equivalent in intellectual practice involves showing your work by talking or writing as thoughts occur to you.
Stream of consciousness writing is where you try and write down what thoughts you're having as you're having them. It's an exercise I used to get assigned in writing classes. Sometimes you'll see it in fiction, as a narrator will use a more associative or chaotic description through a character's internal monologue.
Stream of consciousness (abbreviated to SoC) is hard to do well. Most people who sit down to just write things will write broken grammar or repetitive sentences. Scott Alexander excepted, people generally don't look over their unfiltered thoughts and think "yeah, that looks good, publish it."
People get closer to a stream of consciousness when they speak aloud. Public speaking teachers are constantly exhorting their students to remove the ums and ers and other filler words from their speech, but we still do a lot of "wait, hang on, I mean this instead" or worse, unnoticed self-contradictions when we're talking without a rehearsed script.
We catch our own mistakes, given longer to review ourselves.
Knowing this, some people (myself included) are reluctant to relax into SoC. I think this is usually correct, and specifically incorrect when trying to learn a new mental skill.
I used to be annoyed at having to show my work in math class. Often I would work out an answer by intuitive mental hacks, infrequently by rote memorization, and depending on the format by eyeballing the multiple choice answers and guessing. When that happened, I was reluctant to painstakingly write down my steps on the page. It was even more annoying when I had the right final answer, and still lost points from mistakes in my work.
But how else is the teacher to make good adjustments to my technique?
When I was learning to throw a punch, my motion was very visible to the instructor. Same with improving my tennis serve, my fencing lunge, and my posture when taking massage therapy classes. For physical skills it's natural for whoever is teaching you to have a decent ability to notice when you're doing something wrong mid-motion, not just by checking if the end result works or not.
You could learn tennis entirely by iterating to do more of what worked and less of what didn't work, but tennis has the advantage of a very quick feedback loop - you can watch where the ball went after each hit and whether that was where you meant to send it. For massage, I quickly figured out what pressures felt nice or felt bad from the practice patient speaking up, but it would have taken me a long time to figure out a better posture which wouldn't give me a crick in my neck just by trial and error.
Thus an advantage to having a teacher who can spot the small interim mistakes in progress.
Have you tried just doing it?
The pressure against SoC for me is not wanting to make mistakes in front of people, so I don't make mistakes. I journal in a book nobody else gets to read, I sit down in front of the keyboard and just type things that come into my mind then delete it when I'm done, I talk out loud into a mic and play my voice back. All of these are ways to access my stream of consciousness.
And then - and this is important - I don't beat myself up about mistakes there. I try and be less wrong in future, yes. I'll notice patterns like "huh, I do seem to go in loops a lot, maybe I should try and stop after the second time around" or "I notice I'm bringing up that I want two things, and they're incompatible, and I'm not bringing up which of the two I want more. What's up with that?" That's the kind of thing that can be very apparent when I look at a page or two of text I've just typed out by disconnecting the filter that's usually between my brain and my keyboard, but I'd never notice with my thoughts written on water.
Moving away from self practice, I notice that many of my teachers have been doing this exact thing. In university and in the better high school classes, I was sometimes asked to just talk out loud about how I was trying to solve a particular problem, and the teacher would gently intercede on some trains of thought or make encouraging sounds on others. There is good form for thoughts.
When I had this notion, I asked an acquaintance of mine who was in debate club if his instructors ever interrupted him mid-debate to give commentary. He said yes. That's not a slam dunk in favour of the advantages of SoC since "mid-debate" has elements of performance, not just thinking, but it's a nudge that way.
Rationalist groups can provide feedback on better thinking under the right circumstances. When I first started attending regular meetups in Boston, I mostly spectated arguments between a few of the old hands. The way they tracked cruxes or stated evidentials was relatively new to me. Gradually I started to talk more, and in the process of conversation got corrected more. You do need a few people who know what they're doing for this to work, but the osmosis approach to teaching rationality isn't completely hopeless.
Circling back to self practice, I'm cautiously optimistic about LLMs here. It feels like with the right prompting there could be a way to have the machine check stream of consciousness in a way that didn't feel embarrassing and judged the way I would feel if I showed someone else my journal.
All sorts of things could be built upon this scaffold.
I think a lot of painstakingly slow metis transfer can be sped up by thinking out loud. When I'm working with someone else who has more skill than me in some domain, I often encourage them to interrupt me as I think aloud in conversation with them. It's great for catching mistakes at the earliest part of the path.
Lots of positive mental habits and trainings rely on an honest scaffold of your thoughts. Cognitive behavioral therapy, noticing negative spirals, starting to use evidentials, practicing fermi estimates, single crux, all of these work best when you have some access to what your brain is doing at a level lower than the composed careful words coming out of your mouth.
Teachers, my suggestion is to add this tool to your toolkit. It's okay to ask your students for a stream of consciousness. It may take a little work to get them started, and it's not a cure-all, but it's useful in the right places.
And for everyone else; give it a try. Open a blank text document on one side of your screen, open a problem you're working on over in the other side of your screen, and start typing.