For the goal of maintaining the safety and autonomy of particular agents (and potential moral patients), there seems to be something important about maintaining the membranes and boundaries of those agents. A few examples:

A bacterium uses its membrane to protect its internal processes from external influences.

AI generated image: microscope view of a amoeba

A nation maintains its sovereignty by defending its borders.


A human protects their mental integrity by selectively filtering the information that comes in and out of their mind.

AI generated image: a brain surrounded by a fence

What is the membrane of an agent?

Membranes are the things that maintain themselves despite environmental perturbation. (This is one reason I prefer the term “membranes” over “boundaries”. I think agent membranes are best understood as being living.)

Membranes are one way that agents can causally distance themselves from their environment and threats. 

Think of a bacterium: it has a membrane that separates it from its environment, and the bacterium can exert a great deal of control on what comes in and what goes out using its membrane (via ion channels, gap junctions, etc.).


The bacterium uses its membrane to attempt to keep bad things (e.g.: toxins, potential pathogens) out, and keep good things (e.g.: energy, sovereignty) in.

When an agent fails to causally distance themselves from their environment and threats, it dies.

When an agent maintains causal distance from its environment— when that agent is able to control how the environment affects it, and also how it affects the environment— then it tends to live.

Agents that persist through time tend to maintain membranes.

Causal distance

If you’re an agent, one way to keep yourself safe from threats is to put physical distance between you and the potential threats. It’s hard to get shot by other people when you’re a thousand miles away from everyone else.

Physical distance can therefore often provide causal distance.


Similarly, membranes can also provide causal distance.


In the absence of physical distance from threats, membranes can help provide causal distance.

Again, consider a bacterium: a bacterium has a membrane that separates it from its environment.


Through this membrane, the bacterium can exert a significant amount of control on what comes in and what goes out (via ion channels, gap junctions, etc.).

“Causal distance” can also be useful for conceptualizing interactions in information space. For example, computer systems can be constructed such that only the processes that need to have read and/or write access to particular resources have the privilege[1] to do so. These privileges can be enforced with cryptography/encryption.


Such a computer system would be secure to the extent that it verifies the information that is received and signs the information that it sent. It’s a computational membrane.

Embedded agents attempt to de-embed themselves

Membranes are one way that embedded agents can try to de-embed themselves from their environment.[2]

For example: I would love to be physically bulletproof, have a perfect immune system, etc. (as long as there were no downsides). Wouldn’t you?

Membranes are not about preferences

Common misunderstanding: "boundaries are just preferences".

Correction: Boundaries/membranes do not necessarily depend on preferences. 

Andrew Critch, «Boundaries» Sequence, Part 3b:

my goal is to treat boundaries as more fundamental than preferences, rather than as merely a feature of them.  In other words, I think boundaries are probably better able to carve reality at the joints than either preferences or utility functions, for the purpose of creating a good working relationship between humanity and AI technology

Formalize "safety"?

See Agent membranes and formalizing “safety” 

Mathematical formalism

How might membranes/boundaries be formalized mathematically?

Markov blankets seem to be a fitting abstraction. 

A Markov blanket between agent and its environment:

Notice that there are no arrows directly between the agent and its environment. Ideally, all influence from one to the other flows through the boundary/membrane (e.g.: your skin).

In which case,

  • Infiltration of information across this Markov blanket measures membrane piercing, and low infiltration indicates the absence of such piercing.
  • (And it may also be useful to keep track of exfiltration across the Markov blanket?[3])

For more details, see distillation Formalizing «Boundaries» with Markov blankets

Also, there are probably other information-theoretic measures that are useful for formalizing membranes/boundaries.

Subscribe to the boundaries/membranes LessWrong tag to get notified of new developments.

Thanks to Jonathan Ng, Alexander Gietelink Oldenziel, Alex Zhu, and Evan Miyazono for reviewing drafts of this post. Thanks to Scott Garrabrant for a conversation that got me thinking about causal distance again. 

  1. ^

    See Principle of least privilege - Wikipedia, a.k.a. Principle of Least Authority

  2. ^

    Something to think about: relationship to instrumental convergence? ~“Embedded agents want to become de-embedded agents.”

  3. ^

    exfiltration, i.e.: privacy and the absence of mind-reading. But I need to think more about this. Related section: “Maintaining Boundaries is about Maintaining Free Will and Privacy” by Scott Garrabrant.

New to LessWrong?

New Comment
3 comments, sorted by Click to highlight new comments since: Today at 4:48 PM

Microplastics (and pollution - both mimetic and actual) wreck boundaries by intercalcating between boundaries/cell membranes and reducing the integrity of the boundary. To reinforce proper boundaries, it's important to maintain the organism's overall health (eg deuterated PUFAs like RT-011 help reduce oxidative stress on polyunsaturated fatty acids in the cell membrane).

[when the integrity of boundaries is weakened, the organism's channel capacity is reduced by the extra noise].


For an organism to have healthy boundaries/Markov blanket (within both its cells and organ systems [and also between DMN and FPN networks of the brain]), organs must also compartmentalize their own compute shielded from influences that disrupt their compute. 

Karl Friston often insulates his compute from that of the world, and this makes him act more as an independent thinker. I often wonder if extremely effective people (eg Andrej Karpathy) have stronger agent membranes than others (though the process of aging dissolves boundaries - "death is what happens when the rest of the environment has full predictive power over the agent).

There are many layers of Markov blankets/boundaries and we should be doing a better job of communicating this to example thinkers rather than just to rule thinkers.

(it will be interesting to see if BCIs/t-FUS reinforce or dissolve Markov boundaries - they can help denoise the brain [esp from default mode noise], but the act of inserting a BCI can disrupt physical boundaries)

Nice post.

"Membranes are one way that embedded agents can try to de-embed themselves from their environment."

I would like to hear more elaboration on "de-embedding". For agents who which are embedded in and interact directly with the physical world, I'm not sure that a process of de-embedding is well defined.

There are fundamental thermodynamic properties of agents that are relevant here. Discussion of agent membranes could also include an analysis of how the environment and agent do work on each other via the mebrane, and how the agent dissipates waste heat and excess entropy to the environment. 

De-embedding, as in, making your membrane as strong as possible. More technically: minimizing infiltration as much as possible in response to potential threats across your Markov blanket