"To me, it feels viscerally like I have the whole argument in mind, but when I look closely, it's obviously not the case. I'm just boldly going on and putting faith in my memory system to provide the next pieces when I need them. And usually it works out."
This closely relates to the kind of experience that makes me think about language as post hoc symbolic logic fitting to the neural computations of the brain. Which kinda inspired the hypothesis of a language model trained on a distinct neural net being similar to how humans experience consciousness (and gives the illusion of free will).
My original idea (and great points against the intuition by Rohin)
The next time you are making a complicated argument, if you can, try and watch yourself recalling bits and pieces at a time. To me, it feels viscerally like I have the whole argument in mind, but when I look closely, it's obviously not the case. I'm just boldly going on and putting faith in my memory system to provide the next pieces when I need them. And usually it works out.
Yes! And, I would offer an additional, alternative way of phrasing this: "you" actually do have the whole argument in mind, but it's a higher-level "you", a slower but more inclusive one, corresponding to a higher level of memory caching.
(When it doesn't, there's this whole failure mode where people continue viscerally feeling like they can make the argument, even though they don't have the pieces; and I think this is where a lot of bad reasoning comes from.)
The problem here ^ then becomes becomes a problem of maintaining appropriate relationships among the different self-layers.
I'm not going to argue against focusing on your breathing, but I don't think this is what I'm getting at. Iirc I have not had an experience of pain or meditation create the kind of reflective moment I'm talking about with the longer-term thing. It's more like when you've been working at a place for a while but you suddenly look around and think "wow, I'm a ___" (doctor, teacher, etc) in a way you didn't before. You can try to have that thought on purpose, but I think that's different from what I'm talking about.
I don't like the "self awareness" framing, because attention/awareness is always on specific things, and "the self" isn't really specific. It's not like there's this dimension "how aware are you of your self right now"; what does that even mean?
Epistemic status: fake psychology, no attempt to be correct in details, but maybe something broadly like this happens
Ever had the experience of laughing at something, and then a few minutes later it sort of hits you again and you start laughing again?
Or forget someone's name you just used, and say "it'll come to me in a minute" (and it does)?
Humans have very limited working memory. But you can sort of simulate a larger working memory by looping over things, "rotating" them in and out.
Computer systems tend to have multiple layers of caches -- SRAM caches DRAM, DRAM caches stuff on the disk. It creates this pyramid of memories, on a spectrum from fast&small to slow&large.
Computers have their own sophisticated ways of managing these memories, but for humans it might make sense to randomly/cyclically rotate stuff in and out (weighted by some measure of importance and relevance) in order to give you a good chance of spotting any important connections between, say, 20 things even though you can only keep 7 things in working memory at once.
I mean, it's very sophisticated. The next time you are making a complicated argument, if you can, try and watch yourself recalling bits and pieces at a time. To me, it feels viscerally like I have the whole argument in mind, but when I look closely, it's obviously not the case. I'm just boldly going on and putting faith in my memory system to provide the next pieces when I need them. And usually it works out.
(When it doesn't, there's this whole failure mode where people continue viscerally feeling like they can make the argument, even though they don't have the pieces; and I think this is where a lot of bad reasoning comes from.)
But there's also this random/cyclical element to it, with things popping in and out for no good reason other than that I haven't thought about it in a bit.
There's also a moderately similar-feeling thing that happens on a longer time scale, which might be a different mechanism. It feels like I have pieces of me which "wake up" periodically and look around. There's the piece that wakes up every few minutes to check whether I'm doing roughly what I want to be doing. There's a piece that wakes up every few hours to really check. There's a piece that wakes up every few days to see whether things are looking ok on that scale. And so on, for a lot of time scales. I've noticed that I have a "mini mid-life crisis" every two years, where I really look around and take stock.
I mean, what I'm describing could be that memories from those time scales come up approximately that often, putting the current moment in context and causing me to consider things with seeming clarity. I talk in terms of "waking up" because it seems like I'm seeing with fresh eyes every so often, but in a fractal way; the fresher, the rarer.
Point being, it generally feels like I can access the full context (my whole life of memories). But of course I can't fit all that in working memory, so I'm not actively processing it, so I'm like a person stumbling around in the dark (relatively speaking) but thinking the lights are on. Every so often, lightning strikes, and the moment of clarity shows the previous stumbling for what it was.
Spaced-repetition learning feels like teaching successively higher selves. When I first look at a flashcard, it only registers for the self-of-the-moment; it vanishes, possibly just a few seconds later. The spaced repetition algorithm gets higher selves to wake up and look at it, putting it in their longer-term memory.