All of Chipmonk's Comments + Replies

IME the process outlined in this book is absolutely right. However, one piece of the framework seems weird to me: it seems to suggests that emotions are the cause of schemas/beliefs:

Intense emotions generate unconscious predictive models of how the world functions and what caused those emotions to occur. The brain then uses those models to guide our future behavior.

This doesn't match my intuitions… imo, more like this: beliefs are the cause of emotions. Where do 'intense emotions' come from?

Maybe emotions are best thought as doing 'prioritization' or serving other essential function, but I don't think they're the bottom

But I don't get why we don't view an observation of events we caused, which another person doesn't like, as an attack.

I think for an individual who is skilled enough about what stuff is theirs (what stuff they (and only they) can observe; what stuff they (and only they) can control), and what stuff isn't theirs, then they would not register this as an attack. 

In practice, though, I think many people are bad at this skill, and so when they hear someone say "(You're doing a bad thing!)", they interpret this not as just a remark that is coming from somew... (read more)

Thank you for compiling this! I've been working with a counselor for the past year using basically this method with great results, though I've only found this book and post today. Great to see support for the idea

Related (H/T @Roman Leventov - comment): 

Dalton Sakthivadivel showed here that boundaries (i.e., sparse couplings) do exist and are "ubiqutuous" in high-dimensional (i.e., complex) systems.

I think that boundaries […] is an undeniably important concept that is usable for inferring ethical behaviour. But I don't think a simple "winning" deontology is derivable from this concept.

I see

I'm currently preparing an article where I describe that from the AI engineering perspective, deontology, virtue ethics, and consequentialism

please lmk when you post this. i've subscribed to your lw posts too


FWIW, I don't think the examples given necessarily break «membranes» as a "winning" deontological theory.

A surgeon intruding into the boundaries of a patient i

... (read more)

Okay, I'll try to summarize your main points. Please let me know if this is right

  1. You think «membranes» will not be able to be formalized in a consistent way, especially in a way that is consistent across different levels of modeling
  2. "It seems easy to find counterexamples when intruding into someone's boundaries is an ethical thing to do and obtaining from that would be highly unethical."

Have I missed anything? I'll respond after you confirm.

Also, would you please share any key example(s) of #2?

1Roman Leventov6d
No, I think membranes could be formalised (Markov blankets, objective "joints" of the environment as in https://arxiv.org/abs/2303.01514, [https://arxiv.org/abs/2303.01514,] etc.; though theory-laden, I think that the "diff" between the boundaries identifiable from the perspective of different theories is usually negligible). We, humans, intrude into each others' boundaries, boundaries of animals, organisations, communities, etc. all the time. A surgeon intruding into the boundaries of a patient is an ethical thing to do. If AI automated the entire economy, then waited until humanity completely loses the ability to run the civilisation on their own, and then suddenly stopped any maintenance of the automated systems that support the lives of humans and sees how humans die out because they cannot support themselves would be "respecting humans' boundaries", but would also be an evil treacherous turn. Messing with Hitler's boundaries (i.e., killing him) in 1940 would be an ethical action from the perspective of most systems that may care about that (individual humans, organisations, countries, communities).  I think that boundaries (including consciousness boundaries: what is the locus of animal consciousness? Just the brain or the whole body, or it even extends beyond the body? What is the locus of AI's consciousness?) is an undeniably important concept that is usable for inferring ethical behaviour. But I don't think a simple "winning" deontology is derivable from this concept. I'm currently preparing an article where I describe that from the AI engineering perspective, deontology, virtue ethics, and consequentialism could be seen as engineering techniques (approaches) that could help to produce and continuously infer the ethical style of behaviour. None of these "classical" approaches to normative ethics is either necessary or sufficient, but they all could help to improve the ethics in some cognitive architectures.

Do you think this relates to «Boundaries» for formalizing a bare-bones morality ?

this world where people successfully adapt to superintelligent AI services is a totalitarian police state

& Davidad's Night Watchman

3Charlie Steiner10d
It's related in that you're all talking about maintaining some parts of the status quo, but I think the instrumental technologies (human-directed services vs. agential AIs that directly care about maintaining status-quo boundaries) are pretty different, as are all the arguments related to those technologies.

Some number of the examples in this post don't make sense to me. For example, where is the membrane in "work/life balance"? Or, where is the membrane in "personal space" (see Duncan's post, which is linked).

I think there's a thing that is "social boundaries", which is like preferences— and there's also a thing like "informational or physical membranes", which happens to use the same word "boundaries", but is much more universal than preferences. Personally, I think these two things are worth regarding as separate concepts.

Personally, I like to think about ... (read more)

I realize that none of my posts so far have expressed why I'm so excited about «membranes/boundaries»; I haven't spoken about the problems that this has actually helped me with.

It's not so surprising: I've been writing that post for a long time (since January, actually), and I've had a few major rewrites in the past 6 weeks alone. I'm glad that I've written some general «membranes» content already, but I think it's time to focus on writing the real post.

FWIW, I think a morality based on minimizing «membrane/boundary»[1] violations could possibly avoid the issues outlined here. That is, a form of deontology where the rule is ~"respect the «membranes/boundaries» of sovereign agents". (And I think this works because I think «membranes/boundaries» are universally observable.)

Relevant posts: 

... (read more)
4William D'Alessandro12d
Glad to have this flagged here, thanks. As I've said to @Chipmonk [https://www.lesswrong.com/users/chipmonk?mention=user] privately, I think this sort of boundaries-based deontology shares lots of DNA with the libertarian deontology tradition, which I gestured at in the last footnote. (See https://plato.stanford.edu/entries/ethics-deontological/#PatCenDeoThe [https://plato.stanford.edu/entries/ethics-deontological/#PatCenDeoThe] for an overview.) Philosophers have been discussing this stuff at least since Nozick in the 1970s, so there's lots of sophisticated material to draw on -- I'd encourage boundaries/membranes fans to look at this literature before trying to reinvent everything from scratch.  The SEP article on republicanism also has some nice discussion of conceptual questions about non-interference and non-domination  (https://plato.stanford.edu/entries/republicanism [https://plato.stanford.edu/entries/republicanism]), which I think any approach along these lines will have to grapple with. @Andrew_Critch [https://www.lesswrong.com/users/andrew_critch?mention=user] and @davidad [https://www.lesswrong.com/users/davidad?mention=user], I'd be interested in hearing more about your respective boundaritarian versions of deontology, especially with respect to AI safety applications!

To be clear, I think the bare-bones morality in that post comes from "observe boundaries and then try not to violate them" (or, in Davidad's case: and then proactively defend them (which is stronger)). 

I'll need to think about the rest of your comment more, hm. If you think of examples please lmk:)
Also, wdym that a logical dependency could be itself a membrane? Eg?

One thing— I think the «membranes/boundaries» generator would probably reject the "you get to choose worlds" premise, for that choice is not within your «membrane/boundary». Instead, there's... (read more)

oooo, that's interesting! maximal abstraction… *membranes all the way down...*

I do like this, I'll have to think about it more. Another thing this makes me realize that I like about this is that it requires no privileged perspective: you don't actually know that anything is an agent (just like reality— solipism— etc.)— all you know is that there are membranes and things you can't control…

I already think that trying to control fate is a membrane/boundary violation just as trying to control another sovereign agent is a membrane/boundary violation, and this w... (read more)

Oh, huh. I'm not sure that's in the scope I mean with «membranes/boundaries»

I do think that some people mean something more like an actively defended border when talking about their social boundaries

To be clear, I don't think social requirements and membranes are unrelated. Actually, I think for anyone who is sufficiently conscious, if you violate (or attempt to violate) their «membranes/boundaries», then they will treat that as a violation of their social requirements (colloquial 'boundaries'). In the limit of courage, there is that convergence. 

I think they are trying to say that they have drawn social lines in the sand, wh

... (read more)

Just in case you haven't seen it: «Boundaries/Membranes» and AI safety compilation, «Boundaries» for formalizing a bare-bones morality. But you seem to be talking about this as a membrane insulating cognition, which is something I haven't thought of before... it's an interesting idea i think, i don't know what to make of it. Do let me know if you get more thoughts on it:)

2Vladimir_Nesov19d
Cognition is the membrane, its sanity and alignment insulating the physical world, its capability providing the option of having scarier things pass through. It's an example of membrane vs. boundary distinction, because the membrane is a physical machine, the AI, not some line in the sand. And if it lets through what it shouldn't, the world dies (metaphorically for the world, literally for the people in it), so there is reason to maintain it in good condition. But it's a weird example, because the other side of the membrane looks into the platonic realm, not into another physical location, and it selectively lets through ideas/designs/behaviors, not physical compounds. An analogous example would be a radio, a device made out of atoms that selectively listens to electromagnetic signals. The proposed alignment technique is guarding against hallucinations on the level of chatbot's personality rather than only of facts it voices, avoiding masks that have fictional personalities with fictional values. Not making up values strengthens the prior towards human values.

I intend to take a closer look at this later, but I think you'd be interested in «Boundaries» for formalizing a bare-bones morality, where I talk about the uses of «boundaries/membranes» for creating a consistent-and-also-not-arbitrary form of deontology. And I'd be very interested in hearing your takes on it

Update: see my new comment

but it sounds a lot like what is between a system and its environment

yes. and i think i particularly mean this to mean only boundaries that are 'natural' in some way. probably homeostatic/self-maintaining.

Looked at ChatGPT blurb- Yes this seems extremely related. Thank you, I wasn't aware of his work and i'll have to look into it! Let me know if you do think of any good resources

these "differences" might be the rules, norms, or social structures that separate one social system

this i might disagree with a little. Ie: I wouldn't call the "difference" of a ce... (read more)

5Gunnar_Zarncke19d
A quick summary of Luhmann's systems theory can be found on his Wikipedia page: https://en.wikipedia.org/wiki/Niklas_Luhmann#Systems_theory [https://en.wikipedia.org/wiki/Niklas_Luhmann#Systems_theory]  I recommend his seminal work Social Systems [https://www.goodreads.com/book/show/337482.Social_Systems], though I can't say how good the English translation is. I have read the German original, and it is difficult to read because its approach to its quite abstract subject is to explain and explore from many angles and refer to the body of prior work and their terminology without the benefit of math (though math wouldn't help much I guess at this stage of the field). 
1Chipmonk18d
Today I've added a link to my current research questions for «membranes/boundaries» [https://www.lesswrong.com/posts/HhbwCYmMAnrapwzfX/my-current-research-questions-for-membranes-boundaries], and also fixed the headers
1Chipmonk20d
Today I've slightly updated the post to reflect what I think will be less-confusing terminology for this concept going forward [https://www.lesswrong.com/posts/fDk9hLDpjeT9gZH6h/membranes-is-better-terminology-than-boundaries-alone]. 
1Elizabeth1mo
What an asshole way to ask that question. Participants represented themselves as wanting treatment. It turns out many did not want treatment enough to go through the effort to get it. I was surprised by the extent of this phenomenon but not its existence, which is why "do people get treatment?" was the experimental question in the first place.

The happiest people were the ones who got their results over email, which is a HIPAA violation the company claims to have done by accident

this might not be true? it seems ok if patient requests

but what does 

our viscera, which has now flowed outside our boundary. 

mean? 

Could you give a concrete example?

The overleaf project linked in the last word of "Why Category Theory" is restricted

the object obviously has a viscera that’s outside the boundary

I'm not following you here— 'viscera' is defined to be what's within the boundary, no?

Also, what does it mean for the viscera to have different 'shapes'?

1Jonas Hallgren1mo
Sorry for not responding earlier; working on a post that goes through related things in more detail.  Well I'm just saying that the red blob goes outside the striped circle. The red blob is our viscera, which has now flowed outside our boundary.  I imagine boundaries being a way of depicting the world so the viscera is the "object" that is in the territory whilst the boundary is our map of that object, meaning that the viscera can change without the boundary changing, which in turn leads us to a mismatch and both exfiltration and infiltration in this case.
3awg1mo
Chiming in to also say that I think this post is valuable. But not only this post--posts like this in general. I really appreciate the work of you and people like you who are able to take a complex topic explained across multiple posts/sequences and by multiple people and distill it into a concise summary that feels approachable and understandable while also giving relevant links to find deeper material. Like I think this is really, extremely valuable. So thank you and I look forward to reading more from you and anyone else who wants to submit work like this here!

Bug: I can make myself a co-author on a draft that I've created (a second co-author).

I've compiled all of the current «Boundaries» x AI safety thinking and research (like this post) that I could find here: «Boundaries» and AI safety compilation.

(E.g.: Davidad connected this post to moral patienthood on twitter)

I've compiled all of the current «Boundaries» x AI safety thinking and research (like this post) that I could find here: «Boundaries» and AI safety compilation

I've compiled all of the current «Boundaries» x AI safety thinking and research I could find in this post: «Boundaries» and AI safety compilation. Also see: «Boundaries» for formalizing a bare-bones morality which relates to scoped consequentialism

  • Deontic Sufficiency Hypothesis: There exists a human-understandable set of features of finite trajectories in such a world-model, taking values in , such that we can be reasonably confident that all these features being near 0 implies high probability of existential safety, and such that saturating them at 0 is feasible[2] with high probability, using scientifically-accessible technologies.
... (read more)

I've compiled most if not all of everything Davidad has said about «boundaries» (which are mentioned in this post insofar as "deontic feasibility hypothesis" and "elicitors") to date here: «Boundaries and AI safety compilation. Also see: «Boundaries» for formalizing a bare-bones morality

Ok, I will rename the tag from "«Boundaries»" to "Boundaries [technical]". Fwiw I consider both strings as referring to the same concept, but I see how it might be weird to use «».

I believe I'm abiding by the definition inherent to his sequence, but anyone is free to convince me otherwise.

(Please also let me know if I've violated some norm about naming conventions.)

I've decided to use "«boundaries»" instead of "boundaries" because "boundaries" colloquially refers to something that's more like "Hey you crossed my boundaries, you're so mean!" (see this post for examples), and while I think that these two concepts are related, I find them extraordinarily confusing to consider simultaneously (because "crossing 'boundaries'" does not imp... (read more)

2Raemon1mo
LW is somewhat opinionated about how to do tags. (This doesn't mean there's a hard-and-fast-rule, just that when we're evaluating what makes good tags and considering whether to re-organize tags, the mods reflect on the entire experience of the LW userbase). Generally, we want tags that are "neither too narrow nor too broad".  In this case, if there were other people writing about boundaries-in-a-technical-sense which for some reason was notably different from Critch's definition, and there were some people (maybe just Critch, maybe Critch-plus-a-few-collaborators) who specifically wanted to focus on his definition, then having two tags would make sense. By guess is that anyone writing about boundaries-in-a-technical-sense would end up with a definition similar to Critch's, and there should be just be one tag for all similar work, and the '«' symbol doesn't make sense for the tag.

Here are some more posts which might be also related, but less obviously so. I will leave them in this comment for now, but feel free to argue me into including or excluding any of these.

... (read more)

[APPRENTICE]

I'm looking for someone to mentor me specifically w.r.t. «Boundaries» (or, similarly: Cartesian Frames). I'm interested in this both for AI safety (I have a draft compilation post on this that I will be posting in the next few days, or else I'd share it here), and also as a rationality technique. I'm interested in doing research on and/or distillation for this.

Cartesian boundaries are not real

I disagree with this. This has recently been formalized in Andrew Critch's «Boundaries» Sequence. E.g.: «Boundaries», Part 3a: Defining boundaries as directed Markov blankets.

Boundaries include things like a cell membrane, a fence around yard, and a national border; see Part 1.  In short, a boundary is going to be something that separates the inside of a living system from the outside of the system. More fundamentally, a living system or organism will be defined as 

  • a) a part of the world, with
  • b) a subsystem called
... (read more)

Thanks for writing this! This is very closely related to Andrew Critch's «Boundaries» Sequence, 2022. Part 3a formalizes boundaries in terms of Markov blankets, and leakage in terms of conditional mutual information.

I've also expanded on such leakage ("infiltration and exfiltration") in my post my conceptualizations of infiltration and exfiltration from the «Boundaries» Sequence

I updated the post to add two more examples of exfiltration: one pertaining to BATNAs, and one pertaining to energy/heat loss.

And I added a visualization of agents as blobs.

Maintaining Boundaries is about Maintaining Free Will and Privacy

I really like this conceptualization! Especially "privacy". I've written a post about the finer details of this wrt to infiltration and exfiltration

Here’s a peak at how I summarize this at the end:

  • Infiltration — sovereignty
    • “One should try not to be controlled by others”
    • “One should try not to control others”
  • Exfiltration — privacy / mindreading
    • ~“One should maintain privacy by default and try not to be mind-read by others”
    • ~“One shouldn't speculate about what others are thinking” / “O
... (read more)

I think your comment is mostly relevant and lays out, mechanistically, how speculating about what someone else is thinking can lead to trying to control them (a sovereignty violation); i.e.: from exfiltration to infiltration.

Also—

I was having trouble understanding infiltration and exfiltration conceptually, but I think I made some progress and I wrote a post about my conceptualizations.

Here’s a peak at how I summarize this at the end:

  • Infiltration — sovereignty
    • “One should try not to be controlled by others”
    • “One should try not to control others”
  • Exfiltration — privacy / mindreading
    • ~“One should maintain privacy by default and try not to be mind-read by others”
    • ~“One shouldn't speculate about what others are thinking” / “One shouldn't invade others' privacy”

1. Expansive thinking

Consider a fictional person named Alex. A "job" for Alex is a scope of affairs (features of the world) that Alex is considered responsible for observing and handling.  Alex might have multiple roles that we can think of as jobs, e.g. "office manager", "husband", "neighbor".

I somewhat disagree with how this section is presented so I wrote a post about it and proposed a compromise. 

In summary:

  • I propose defining boundaries in the Alex example not in terms of “jobs”, but in terms of: 1) contracts (mutual agreements between two pa
... (read more)