Comment Permalink
116
These are very good questions. First, two general clarifications:
A. «Boundaries» are not partitions of physical space; they are partitions of a causal graphical model that is an abstraction over the concrete physical world-model.
B. To "pierce" a «boundary» is to counterfactually (with respect to the concrete physical world-model) cause the abstract model that represents the boundary to increase in prediction error (relative to the best augmented abstraction that uses the same state-space factorization but permits arbitrary causal dependencies crossing the boundary).
So, to your particular cases:
- Probably not. There is no fundamental difference between sound and contact. Rather, the fundamental difference is between the usual flow of information through the senses and other flows of information that are possible in the concrete physical world-model but not represented in the abstraction. An interaction that pierces the membrane is one which breaks the abstraction barrier of perception. Ordinary speech acts do not. Only sounds which cause damage (internal state changes that are not well-modelled as mental states) or which otherwise exceed the "operating conditions" in the state space of the «boundary» layer (e.g. certain kinds of superstimuli) would pierce the «boundary».
- Almost surely not. This is why, as an agenda for AI safety, it will be necessary to specify a handful of constructive goals, such as provision of clean water and sustenance and the maintenance of hospitable atmospheric conditions, in addition to the «boundary»-based safety prohibitions.
- Definitely not. Omission of beneficial actions is not a counterfactual impact.
- Probably. This causes prediction error because the abstraction of typical human spatial positions is that they have substantial ability to affect their position between nearby streets by simple locomotory action sequences. But if a human is already effectively imprisoned, then adding more concrete would not create additional/counterfactual prediction error.
- Probably not. Provision of resources (that are within "operating conditions", i.e. not "out-of-distribution") is not a «boundary» violation as long as the human has the typical amount of control of whether to accept them.
- Definitely not. Exploiting behavioural tendencies which are not counterfactually corrupted is not a «boundary» violation.
- Maybe. If the ad's effect on decision-making tendencies is well modelled by the abstraction of typical in-distribution human interactions, then using that channel does not violate the «boundary». Unprecedented superstimuli would, but the precedented patterns in advertising are already pretty bad. This is a weak point of the «boundaries» concept, in my view. We need additional criteria for avoiding psychological harm, including superpersuasion. One is simply to forbid autonomous superhuman systems from communicating to humans at all: any proposed actions which can be meaningfully interpreted by sandboxed human-level supervisory AIs as messages with nontrivial semantics could be rejected. Another approach is Mariven's criterion for deception, but applying this criterion requires modelling human mental states as beliefs about the world (which is certainly not 100% scientifically accurate). I would like to see more work here, and more different proposed approaches.
5
4
2
2
1
1
1
It might be possible to formalize what it means for an agent/moral patient to be safe via membranes/boundaries. This post tells one (just one) story for how the membranes idea could be useful for thinking about existential risk and AI safety.
Formalizing “safety” using agent membranes
A few examples:
A natural abstraction for agent safety?
Agent boundaries/membranes seem to be a natural abstraction representing the safety and autonomy of agents.
Maybe the safety of agents could be largely formalized as the preservation of their membranes.
Distinct from preferences!
Boundaries are also cool because they show a way to respect agents without needing to talking about their preferences or utility functions. Andrew Critch has said the following about this idea:
For instance, respecting the boundary of bacterium would probably mean “preserving or not disrupting its membrane” (as opposed to knowing its preferences and satisfying them).
Protecting agents and infrastructure
By formalizing and preserving the important boundaries in the world, we could be in a better position to protect humanity from AI threats. Examples:
This is for safety, not full alignment
Note that this is only about specifying safety, not full alignment.
See: Safety First: safety before full alignment. The deontic sufficiency hypothesis.
Caveats
I don't think the absence of membrane piercing formalizes all of safety, but I think it gets at a good chunk of what "safety" should mean. Future thinking will have to determine what more is required.
What are examples of violations of agent safety that do not involve membrane piercing?
Markov blankets
How might membranes/boundaries be formalized mathematically? Markov blankets seem to be a fitting abstraction.
Diagram:
In which case,
For more details, see distillation Formalizing «Boundaries» with Markov blankets.
Also, there are probably other information-theoretic measures that are useful for formalizing membranes/boundaries.
Protecting agent membranes/boundaries
See: Protecting agent boundaries.
Subscribe to the boundaries/membranes LessWrong tag to get notified of new developments.
Thanks to Jonathan Ng, Alexander Gietelink Oldenziel, Alex Zhu, and Evan Miyazono for reviewing a draft of this post.
exfiltration, i.e.: privacy and the absence of mind-reading. But I need to think more about this. Related section: “Maintaining Boundaries is about Maintaining Free Will and Privacy” by Scott Garrabrant.