meta note: formatting processing models as a conversing explorer-explainer pair may be useful for you as well. ymmv.
This is a great explanation for why "exobrain" (a) has to include nonverbal / functional environment and (b) has to be dealt with if you want to get anything done.
[epistemic status: messy. probably approximately one of the major processes for task carrythrough failure.]
explorer: why do i care about trivial inconveniences? those sound, uh, trivial
speaker: well, most of the time, youre not gonna be thinking shit like this, youre just gonna be running routines
explorer: what? i havent seen a routine
speaker: yeah because a routine is to a first approximation an alternative to you.
explorer: wait i thought i was just, like, me
speaker: nooo, youre a reflective process. youre optimized for figuring out long term shit and whatever. you got other processes for when you just needa walk to the train station or something.
explorer: oh ok hi everyone
everyone: hi
explorer: so can Doer Optimizers do the thing even if its inconvenient?
speaker: technically, yes.
explorer: ...?
speaker: a typical doer process will be pretty low-energy, itll just try to do the thing the way it usually does it and give up if it doesnt work
explorer: why does it do that?
speaker: if it doesnt do that, it will try to load a webpage and encounter something bad happening and wake up six hours later with a dehydration headache and a third of a new linux distro and—as flashbacks to things involving the word 'firmware' appear before your eyes—you will observe that the webpage is still not loaded
explorer: why did it do that?
speaker: well, it got told to do the thing, and then, like, to do the thing it had to do this other thing and...
explorer: oh. whoa.
speaker: yeah.
explorer: what if i want a Doer Process thats high power but doesnt do that
speaker: well, uh, you can have doer processes that are really closely tied to the task. this is hard, because they require you to both Act, Continuously (so, something thats happy doing the same thing over and over) and Check If Youve Gone Out Of Bounds (consider what youre doing and see if its what you wanna be doing)
explorer: okay but how do i do that
speaker: i guess youd need, like, a routine to do the thing and a routine to check if youre doing the thing, where the checker is lighter weight than your full conciousness.
explorer: can i not be in an environment where things are constantly tryign to entrap you into running a doer process paying attention to them?
speaker: well, yes, but if you want to access information youll have to go out into the internet where everyone wants you to pay attention to them and will try to force you into routines of using their product
explorer: this seems bad
speaker: yes
explorer: ...
speaker: we're working on it