tl;dr: Mixing goal-directedness into cognitive processes that are working to truth-seek about possible futures tends to undermine both truth-seeking and effective pursuit of your goals. Cleanly separating them has nice properties.
[Epistemic Status: Mostly empirically observed rather than rigorously justified, but have seen it many times across different people and within myself so fairly confident there's something here, and have some sketch of the process][1]
Minds are composed of circuits/programs/parts which model goal-states. Or, to put another way, have sub-patterns with preferences over how the world should be.
I'd like to draw attention to one axis of this process, which I'll bind to the following words[3]:
Intention is doing something close to unbiased simulation[4], then applying agentic choice only at the current timestep, trusting future yous at future timesteps to run their own agency with their greater information. It accepts everything at the truth-seeking stage of predicting the future, and only decides things that are immediately actionable.
Trying is when you're applying pressure into the process of your world-model rollouts.[5] A couple of subcategories:
Clinging is selectively rolling out possible futures with agentic choice about which futures to spend cycles on, perhaps refusing to model some futures entirely because they don't include the thing you're clinging to. Trauma and psychological unsafety seems to induce this.[6]
Instrumentalizing is how clinging tends to work in a multi-agent setting. It's trying to force through an outcome by using another agent as an means, by fixing down the goal and shaping another's incentive landscape so they're somewhat forced into being a piece in your game. Instrumentalization tends to lead to micro-managed and ineffective collaborators, and can blow up spectacularly if the agent you tried to make into a subagent of your process disagrees hard enough to rebel.
The alternative
As much as possible, enabling clean world-prediction by deeply accepting the way reality is —[7] in the sense of being willing to model everything, even things which seem very bad to you.[8]
Look out for the subtle feeling of something like forcing or rejecting or pushing inside yourself to make something happen, trace how often that leads to issues down the line, and let the updates propagate.
Resolving your clingings and traumas by spending time in contact with the patterns which induce this, and allowing them to integrate or using other therapeutic techniques and, in the meantime, marking these patterns as clearly about you rather than imposing them on others.
Treating the well-being and empowerment of other agents you're collaborating with as an end not just a means, building non-naive trust with them, and handling conflicts by clean BATNA[9]-known negotiation across mutually visible information boundaries.
My impression is groups that operate more by these principles tend to be dramatically more effective and their individuals more vibrant than those which extensively try to shape the world via distorting their world-models.[10]
[Jargon status: This post pretty compressed by a fairly intricate ontology of minds drawn from a bunch of places, I might try a more accessible version down the road as some readers have commented this is not easy to parse.]
The other main input is our self-model, which seems to be able to be quickly polled for black-box "what would I do" intuitions during those roll-outs of possible futures, as well as, in a healthy mind, the ability to deeply reflect on what you would like to do, recruiting much of the world-model to help.
Possibly Monte Carlo-style. Evenly applying the truth-focused update rule to your cleanly epistemic world model with random choices on points where you're uncertain many times, and using the outputs of this as your best-guess representation of the structure of the future.
Trying, in the sense this post uses, seems to damage people's epistemics. This might be because the most straightforward way to apply internal pressure to avoid rolling out bad-looking branches is to rate them as improbable/OK to ignore in a way divorced from epistemics.
Clinging can blind you to possibilities further into the game tree that your agency refused to roll out because they were ranked too bad, and leave you unprepared for things going wrong in ways that were predictable if you let yourself evaluate them.
tl;dr: Mixing goal-directedness into cognitive processes that are working to truth-seek about possible futures tends to undermine both truth-seeking and effective pursuit of your goals. Cleanly separating them has nice properties.
[Epistemic Status: Mostly empirically observed rather than rigorously justified, but have seen it many times across different people and within myself so fairly confident there's something here, and have some sketch of the process][1]
Minds are composed of circuits/programs/parts which model goal-states. Or, to put another way, have sub-patterns with preferences over how the world should be.
We imagine possible futures that could come out of different actions using our world models, and use this prediction of outcome as an input to action selection[2] (including internal actions like choice of thoughts).
I'd like to draw attention to one axis of this process, which I'll bind to the following words[3]:
The alternative
My impression is groups that operate more by these principles tend to be dramatically more effective and their individuals more vibrant than those which extensively try to shape the world via distorting their world-models.[10]
[Jargon status: This post pretty compressed by a fairly intricate ontology of minds drawn from a bunch of places, I might try a more accessible version down the road as some readers have commented this is not easy to parse.]
The other main input is our self-model, which seems to be able to be quickly polled for black-box "what would I do" intuitions during those roll-outs of possible futures, as well as, in a healthy mind, the ability to deeply reflect on what you would like to do, recruiting much of the world-model to help.
I welcome input on choices of terms here, especially from people who know much more about Buddhism than me, as there might already be technical terms.
Possibly Monte Carlo-style. Evenly applying the truth-focused update rule to your cleanly epistemic world model with random choices on points where you're uncertain many times, and using the outputs of this as your best-guess representation of the structure of the future.
Trying, in the sense this post uses, seems to damage people's epistemics. This might be because the most straightforward way to apply internal pressure to avoid rolling out bad-looking branches is to rate them as improbable/OK to ignore in a way divorced from epistemics.
Clinging can blind you to possibilities further into the game tree that your agency refused to roll out because they were ranked too bad, and leave you unprepared for things going wrong in ways that were predictable if you let yourself evaluate them.
No AIs were used in the making of this post, but I did learn how to use em dashes from AI.
This does not mean you don't get to have preferences, just that your world-model should be able to run separately from your preferences.
Best Alternative to a Negotiated Agreement. It represents the course of action which happens if negotiations fail and no agreement is reached.
I think this is why Buddhism is so interested in Equanimity, and what the whole thing around "Don't try" is.