I’ve quite liked Andrew Critch’s «Boundaries» Sequence so far, and I want to share how I personally understand infiltration and exfiltration (concepts defined in «Boundaries», Part 3a: Defining boundaries as directed Markov blankets).
In general, infiltration and exfiltration are about leakage across the boundaries that divide "stuff". And in this case, I consider the leakage in and out of agents just like you.
I will break infiltration and exfiltration down into the four cases— infiltration into oneself, infiltration into others, exfiltration from oneself, and exfiltration from others— and I will share how I think about each more practically.
Here’s a peak at how I summarize this at the end:
- Infiltration — sovereignty
- “One should try not to be controlled by others”
- “One should try not to control others”
- Exfiltration — privacy / mindreading
- ~“One should maintain privacy by default and try not to be mind-read by others”
- ~“One shouldn't speculate about what others are thinking” / “One shouldn't invade others' privacy”
Infiltration
Infiltration into oneself
In order to maintain the integrity of one’s own decision rule, one should be wary of infiltration from others into oneself. “One should seek to minimize infiltration from others into oneself.”
I.e.: “One should not allow oneself to be controlled by others.”
Infiltration into others
“One should minimize infiltration from oneself into the viscera of others.”
For example, if one were to maximize that, then that would be to strip others of their autonomy and (effectively) would kill them. Note: I have more reasons that I won’t explain here. (Here I assume that one wishes to cooperate with other agents by default.)
I.e.: “One should not try to control others.”
What is infiltration really?
For example, what makes a) “asking someone to do something” different from b) “manipulating someone into doing something”?
More generally, what determines whether outside influences are a) going through the boundary to get to the viscera or are b) directly affecting the viscera?
I tentatively think the difference is: whether the outside influence leads to a violation of the integrity of the agent’s decision rule rθ.
I.e.: One should not allow oneself to be controlled from the outside, because that would supplant the validity of one’s decision rule’, which would result in effective death.
[aside: I suspect this is also closely related to Alfred Adler’s Individual Psychology. I’m looking into this currently.]
Succinctly: infiltration is about direct issues of sovereignty (free will): controlling others, or being controlled by others/the environment.
Exfiltration
Exfiltration is trickier, and I’m more certain in what I've written above than below. Please leave a comment if you have thoughts on how to understand exfiltration better.
Edit: This section was originally written solely in terms of "mindreading", but a few days after I posted this I saw Scott Garabrant's post: Boundaries vs Frames where he connects this all to privacy, and I quite like the connection! I've updated this post in light of that.
Here are some examples that I think are central to exfiltration:
- If other people were to start reading your mind, then the outside world would start behaving in complicated and unwanted ways.
- In general: privacy is important!
- To illustrate the above point, suppose all of the US Government’s communications suddenly became viewable by anyone in real time. And suppose there also existed individuals or groups exploited this. In that case, I imagine that the USG wouldn’t be able to effectively act in many ways it previously could: anyone who opposes the USG’s intentions could simply anticipate its actions and counter them. If anything, I predict the USG would quickly lose relevance as an agent, and rather be more of a mechanical part of the environment (at least for those who exploit it).
- Note also: I suspect that the USG would no longer be able to act if modeling the environment as separate from itself (i.e.: its situation rule breaks).
- Alice and Bob are married. If Alice wants something from Bob, I claim that she should tell him directly, as opposed to, for example, expecting him to notice how the subtle motions of her attention reveal her desires. That is, exfiltration is not intentional communication, and something seems off to me about her expecting him to “read her mind” (to notice the exfiltration of her mindstate/desires into the world).
- Also, Alice is expecting Bob to cross a boundary in this situation, which is odd.
- Let's suppose you run a YouTube channel, and a company approaches you for a sponsorship deal on your next video. You've taken many similar deals in the past, and the conditions and prices of those deals is either public, or private.
- In the world where the details of your previous deals are public (i.e.: perfect exfiltration), the other company will see that you took a similar deal in the past for a price of only $X. If you try to negotiate above that, they probably won't budge. After all, they know your BATNA.
- In the world where the details of your previous sponorship deals are private (no exfiltration, perfect privacy), the company doesn't have that information, and you have a better position in the negotiation.
- Warm-blooded animals losing internal heat is exfiltration of energy. (If *energy* is considered in place of *information*, then heat lost that is lost from the organs of an endotherm is exfiltration.) Animals can and do use surface body fat and other forms of insulation to minimize such exfiltration. [Example inspired by Boundaries enable positive material-informational feedback loops.]
In light of these examples, you, an agent, might adopt a directive like:
- “Maintain privacy by default and try to not be mind-read by others”
Exfiltration from oneself
In order to preserve one’s ability to influence and predict the world, and therefore in order to preserve one’s decision-making ability, one should be wary of exfiltration from oneself to others. “One should seek to minimize exfiltration from oneself to others.”
That is, as an agent, ideally all of your affect on the outside world is happening intentionally (through your designated boundary channels/through the proper interface).
~“you don’t want to be mindread against your will”.
Also: Don't let sensitive stuff eg heat/energy/your guts out. E.g.: Single-celled Organism Dies
Exfiltration from others
In order to respect other agents, one shouldn’t treat exfiltration from others as communication from them.
Perhaps:
- “One shouldn’t try to read others’ minds.”
- “One shouldn’t speculate about what others are thinking.”
- “One should respect the privacy of others.”
I think this is neat— from my personal ontology for «boundaries» (before reading Critch’s sequence), I’ve converged on a directive ~“One should not speculate on what another is thinking”, and that sounds very similar.
I will take this occasion to note that, empirically in my life, unwanted effects occur often when I speculate about what someone else is thinking. (Note that I consider “speculating about what someone else is thinking” to be distinct from being in high-bandwidth communication with that person.)
But why would trying to read others’ minds be bad? One idea I have is: speculating is usually just the step before attempting to control others, and others don’t want to be controlled, so if they think you’re trying to control them (because you're speculating), then they will resist you and even fight you. Speculating is disrespectful.
This is also my main reason to recommend against "infiltration to others": others will fight back.
Also, I'm not sure that there are any good reasons to speculate about what someone else is thinking (without their permission) (excludes: competition; acausal trade).
Also: This reminds me of Scott Garabrant’s “Don’t predict people” / anti-inductivity. (I don’t know where I heard this.)
What is exfiltration really?
For example, what precisely makes a) “directly telling others what you are thinking” different from b) “someone else mindreading you or invading your privacy”?
More generally, what determines whether (~)communication is a) going through the boundary to get to the outside world or is b) directly affecting the outside world?
I tentatively think the difference is: whether the outside influence leads to a violation of the integrity of the agent’s situation rule sn.
I.e.: High exfiltration from an agent would supplant the validity of their situation rule, which would undermine their sovereignty and effectively result in death. {?}
So we have these two things—
- "someone else mindreading you or invading your privacy"
- “directly telling others what you are thinking”
How are they precisely different? I see at least two ways:
- when you say something to someone, you also know that you said it to them. — You’re aware of the information flow.
- when you say something to someone, you’ve chosen what you’ve said and not said. — You’ve approved of the information flow.
So if others mindread you (completely or partially) or are able to invade your privacy, then they get information that 1) you don’t know that they have, and 2) that you might not want them to have.
Then, crucially, the outside world (including other people) starts to act in ways that have a complex relationship with your uncommunicated intentions.
Succinctly: exfiltration is about issues that arise from privacy invasion (eg mindreading), which is then instrumental in further issues of sovereignty.
"Crossing «boundaries»"
Terminology: I would define “crossing «boundaries»” as violating any of these four cases. I would also define “respecting” someone’s boundaries as precisely the opposite.
Visualization: agents are blobs
Once last conceptualization I have to add is how I visualize agents in light of infiltration and exfiltration:
Agents are blobs. And in order to perpetuate their existence, they must do a few things:
- They must hold themselves together, and prevent their "me-ness" from leaking out. They must constantly be pulling themselves together. That is, they must minimize exfiltration.
- Note that their "me-ness" includes atoms, information, energy, etc.
- They must keep foreign entities out, and prevent outside influence from getting in. They must constantly be pushing foreign entities out. They must minimize infiltration.
- They must still interact with the environment, however, and so they establish dedicated interfaces.
- Some examples:
- Food goes into the dedicated food-hole;
- Waste comes out of the dedicated waste-hole;
- Information from other agents comes through dedicated, sandboxed senses (a form of passive boundary);
- Information to other agents comes out through its own intentional channels (a form of active boundary).
- Some examples:
- Last of all, they should probably respect other agents in these same ways (i.e.: "do not try to infiltrate/control others", "do not try to breach privacy/exfiltration of others"), lest they may be defected on by other agents [of similar or greater power as them].
(In another sense, boundaries are about locally fighting entropy.)
Conclusion
That’s how I understand these concepts more conceptually. I hope that helps. Let me know if you disagree. And if you have ideas to add for understanding exfiltration better, please leave a comment or let me know.
Summary
- Infiltration — sovereignty
- “One should try not to be controlled by others”
- “One should try not to control others”
- Exfiltration — privacy / mindreading
- ~“One should maintain privacy by default and try not to be mind-read by others”
- ~“One shouldn't speculate about what others are thinking” / “One shouldn't invade others' privacy”
Final notes
I’m also writing some more posts about «Boundaries» (update: Critiquing/clarifying «Boundaries» Part 2: 1. Expansive Thinking), both about Critch’s sequence, and also from my thinking about this as a social rationality technique. «Boundaries» is the main idea I’ve been thinking about for the past half year.
Also, Critch, if you’re reading this— I’m very interested in contributing to the «boundaries» model.
Not sure if this is relevant, but I was thinking why exactly I hate the idea of others reading my mind...
First, my mind being opaque provides me some slack. There are probably many bugs in my thinking, but if you cannot see them, you cannot exploit them. Of course they may over time become noticeable by observing my behavior, but then there is also a chance that I will notice them and fix them. It's not perfect, but it's better than being completely transparent. Basically, there is an arms race between bad actors trying to exploit my bugs and me trying to fix them; mind-reading provides an advantage for the bad actors. (Though there are situations, for example a session with an aligned therapist or a trusted friend, where temporary transparency is useful.)
It's not just bugs, but also the ability to predict success of blackmailing. If people could run simulations of me and determine in advance which attempts to blackmail me will be successful and which ones will fail, I would expect more attempts to blackmail, because there is a cost to an incorrect guess. Even worse if this could be somehow automated (like, you invent a new method to blackmail people, then ask ChatGPT to scan thousands of people and provide a sorted list of ones who would react most desirably from your perspective).
Second objection is that even if mind-reading is only available under certain circumstances, this would create a motivation for people in power to push people into those circumstances. For example, if we imagine that some people are inherently transparent and others are inherently opaque, the transparent ones would be trusted more. So if there are e.g. two candidates for a job, one transparent and one opaque, many employers would choose to hire the transparent one. (Except for situations where you need the employee to be opaque against a third party, e.g. a salesman. But even, if such thing is possible, you would hope for someone who is transparent against you and opaque against others.) The mentally opaque people would find themselves discriminated against.
On the other hand, suppose that the ability of having your mind read depends on what you do. (Like, there are rituals to make your mind more transparent, and rituals to make your mind more opaque.) There I would expect the people in power to pressure me into making myself more transparent, and punish me if I refuse. And if it is a mixed situation, like the transparency is partially heritable and partially trained, the result would be the same.
This is a reason why it is good to object against mind-reading in general. Being transparent is bad for you, but being opaque in a situation where most people are transparent, could also be bad for you if it is a known fact.
I think your comment is mostly relevant and lays out, mechanistically, how speculating about what someone else is thinking can lead to trying to control them (a sovereignty violation); i.e.: from exfiltration to infiltration.
Also—
I updated the post to add two more examples of exfiltration: one pertaining to BATNAs, and one pertaining to energy/heat loss.
And I added a visualization of agents as blobs.