I've curated this post, for these three reasons (starting with the most important):
My thoughts on extensions and improvements:
---
Personally, this post managed to describe levels of social meta in common knowledge that I had run away scared from trying to explain in my post on the topic (I just quoted someone else's explanation at length). I spent a while trying to think about how to explain its involvement in one-on-one social interactions, and had not come up with anything as concrete and clear (if a little cognitively taxing) as this one.
Also, you mention that you think that the community probably would prefer this quick post than no post at all, and that you think others will be able and happy to think about it + test it for themselves if it seems important. This seems correct to me - when you only have the time for a two-hour post, writing it and giving this context at the head of a post is great, can be very valuable, and I'm really glad you did it. Generally it seems both that folks are very happy you wrote this MVP version, and that they will try to test it out. (To those of you who do try it out, writing your experiences up in a LW post can be really helpful to others!)
Oh, one other thought I had when reading this post: It seems to me that, when trying to come up with good explanations, people reliably don't consider ones that don't fit nicely into working memory. (Some would even say that the very definition of a good explanation is compressing a hypothesis to fit into fewer working memory slots.) But I notice that here it lead to people actually not considering true hypotheses (that more than three meta levels are being used). Perhaps this is merely another point in favour of the 'use external working memory' argument, but I can't help but feel there is some more specific heuristic it recommends about the class of questions where we're likely to make this mistake.
I’m intrigued by the explicit unrolling in contrast to circling. I wonder how much circling is an instance of developing overpowered tools on weird partly-orthogonal dimensions (like embodiment) because you haven’t yet discovered the basic simple structure of the domain.
To what extent is Circling working hard to train capacities that are being used as workarounds for limited working memory and insufficient theoretical understanding the structure of human interaction?
(This is a real question. My guess is, “some, but less than 30%”.)
A lot of my strategies for dealing with situations of this sort are circling-y, and I feel like a lot of that is superfluous. If I had a better theoretical understanding, I could do the thing with much more efficiency.
For instance, I exert a lot of effort to be attuned to the other person in general and to be picking up subtle signs from them, and tracking where they’re at. If had a more correct theoretical understanding, a better ontology, I would only need to be tracking the few things that it turns out are actually relevant.
Since humans don’t know what those factors are, now, people are skilled at this sort of interaction insofar as they can track everything that’s happening with the other person, and as a result, also capture the few things that are relevant to the underlying structure.
I suspect that others disagree strongly with me here.
(Crossposted from here)
Circling is working on a similar problem, and training capacities that are used as workarounds: This feels true to me.
I think this even more visible in the circling-variant called "T-Group," where people tell a short narrative on why they think they're having a described emotional reaction. Very frequently, the explanations encapsulate 1-2 layers of meta, or explicitly gesture at certain/uncertain pieces of common knowledge.
(ex: Joe is responding to Jane's response and feels U, Jane is responding to Jane's interpretation of Joe's response to Jane and feels V. Albert notes X, models that Betty would be troubled by X, and feels concern that there might not be common knowledge of this. Betty believes that there is common knowledge of X, and that everyone feels wary about it. Betty queries if anyone disagrees with that interpretation. Carrie notes that she hadn't initially been aware of X, but is now aware of X, and feels sad and a little scared.)
When I look at it this way, it becomes even clearer why T-Group comes with an exhortation to always include the experiencer as an object in your sentence ("I feel", "I make it mean", "I infer"). If the next person is going to do meta on your meta, it helps if they don't need to recalculate out the layer representing "you," and it's useful to explicitly differentiate between yourself and common knowledge.
(Actually, the more I think about it, the more T-Group looks like a hybrid between circling and explicit modeling. And the fact that they work well together suggests to me that not everything in the circling skillset gets eclipsed when you switch to explicit-modeling.)
T-Group stuff sounds interesting. Does this Wikipedia article refer to the same thing you were talking about?
...huh. I guess I know of one particular variety, and that variety is very self-contained and circling-adjacent (I almost could have called it "Narrative Circling", if that didn't seem like such a contradiction-in-terms). But from the wiki article, T-Group appears to refer to a more nebulous and broad category of things, some of which seem not nearly so self-contained.
The thing I had run into functioned basically as described here (scroll down for the written description). This read to me as clearly cicling-adjacent, and I didn't think all that hard about where it had come from.
The wikipedia description struck me as surprisingly uninformative about the details of the practice itself. But from poking around a bit on the internet just now... I get the impression that T-Group can refer to something similar to what I described, but can also be used to refer to something close to an experimental leadership/decision-making structure that uses the "T-Group" as part of their intragroup conflict-resolution method?
I knew that the variety I had run into was a bit homebrew, and probably had aspects of circling bred into it. I don't think I appreciated just how different it could be from other people's usage of/context for the term. That said, I do see some signs of shared lineage.
The techniques feel related, and the facilitating ethos of awareness, learning, honesty, and goallessness feels similar. But the variety I ran into felt more tightly-defined and compartmentalized, and I was mostly doing it with strangers.
I admit that with a high bar of trust and decently committed participants, I could actually see it working well as a social-information-gathering method? But the idea of being dragged into doing it with coworkers, or of treating it like a primary conflict-resolution technique, seems quite troubling to me.
Alex felt it was bad that Alex felt that Bailey felt that Alex leaving out the milk was bad.
I want to point out that not all instances of the word "felt" mean the same thing here, and I want to split the two meanings into new words for clarity. I think this has consequences about what it means to have feelings about one's own feelings.
Bailey felt that Alex leaving out the milk was bad.
In this case, "feeling" stands for "evaluating". Bailey gives a low value to Alex's action, and in some sense, maybe even to Alex. E.g. Alex is dumb, or Alex's actions are suboptimal.
Alex felt that Bailey felt that Alex leaving out the milk was bad.
In this case, the top level "feeling" stands for "sensing". This is a complex phenomenon:
Alex felt it was bad that Alex felt that Bailey felt that Alex leaving out the milk was bad.
The top level "feeling" stands again for "evaluating". After observing Z in yourself, you give it a low value. But Z is a proxy for X given that X is true (you may be wrong inferring X from Y). Giving a low value to Z roughly means "I wish it was not the case that X", but it's difficult to recognize as such because I think it happens very rapidly and as a complex mental sensation rather than a sentence in your head. I posit that all feelings about one's own feelings are of this kind, and I'm interested in hearing counterexamples.
So, to recap:
Alex evaluated it was bad that Alex sensed that Bailey evaluated that Alex leaving out the milk was bad.
I think the unrolling is clearer this way.
A lot of excellent points in this post. I particularly like the one about having feelings about one's own feelings, which is something I think I've always understood on some level but perhaps the first time I thought about it consciously was from Seth Rogan's line in 40-Year-Old Virgin: "[Y]our depression is boring me for one thing, and it's actually making me a little depressed, which is then in turn making me more depressed that you're actually affecting my mood."
I think posts like this prove a more general point: we humans are thinking on many levels of meta, all the time, and insistence on pulling the levels apart to examine separately isn't "complicating things"; it's just a way of seeing more clearly what's already in our line of vision.
Thanks for the read (honestly, noticed some very interesting points IMO) but I kind of fail to understand what exactly is your claim about the method you introduced.
Are you saying that it is a good model representation of social interaction? If so I would partially agree. Its cool that the model captures all the mental steps all the participants are making (if you bother to completely unroll everything), but it's not computationally superior to saying that: things like calling someone "a downer" are general beliefs that rely on a varying empirical basis, and should be checked for their verity.
In other words - instead of saying that "Bailey is a downer" means "Alex felt it was bad that Alex felt that Bailey felt that Alex doing X was bad, for multiple values of X" you can say that it means "Alex believes he observed Bailey to act as 'a downer' a sufficient amount of times" and unroll that (ask Alex why does he think Bailey's a downer, what does being a downer mean or what does he base her being one on) only if necessary. Most people would understand this explanation more intuitively, in my experience. Some might even find it trivial, cause, for example, if you call someone irresponsible more than once, most of the times you are aware that you think he has acted "irresponsibly" a sufficient amount of times, even if you don't phrase it that exact way. And this explanation is not inferior to yours in the theoretical sense, it doesn't supply less data, and it does seem a little more cost effective, explanation-wise.
Are you saying that it is a good method for practical conflict resolution? It very well may be, but your experience only teaches us that after engaging in a cooperative activity with Alex for a while, he understood that he felt judged and was more inclined to believe you weren't judging him. Psychologically, engaging in a safe activity with someone, even your captor in a hypothetical hostage situation, will diffuse tension and humanize conflicting parties in each other's eyes. It could be that you could have played a short card game with Alex and he would have been more cooperative afterwards all the same.
Even if we claim that analyzing the problem is emotionally helpful in itself, the analysis doesn't have to be all that rigorous, coherent and complete. Many conflict resolution therapy methods focus on giving all parties in a conflict the opportunity to feel heard, which makes it easier to reach emotional catharsis, and therefore agreement. But feeling heard is only a equal to or lesser amount of understanding than being completely, logically understood. Therapy methods such as these may (or may not) be more effective in achieving results, or may be just less effort-demanding.
In order to establish that this method is particularly effective, we either have to get some experimental data showing it gets better outcomes in some criteria, or explain what it has going for it that other methods unequivocally don't.
Anyway, I enjoyed reading this and would be glad if you posted some of your refined conclusions in the future :)
Thank you so much for sharing this concept with me a few years ago, it's still in the top ten of largest coherent boosts another human gave me.
I want to mention, for citation reasons - I am pretty sure a large amount of the discussion about this topic as an explicit, defineable human-common-knowledge came into my friend group through you, despite that I and others have written posts using the concept since you shared it, and that perhaps it was just your retelling of an existing thing. I really appreciate it.
I'm generally in favor of public praise and private criticism, but this post really rubbed me the wrong way. To me it reads as a group of neurotic people getting together to try to get out of neuroticism by being even more neurotic at each other. Or, that in a quest to avoid interacting with the layer of intentions, let's go arbitrarily deep on the recursion stack at the algorithmic/strategy layer of understanding.
Also really bothered by calling a series of reactions spread over time levels of meta. Actually going meta would be paying attention to the structure of the back and forth rather than the individual steps in the back and forth.
Huh, am surprised that this was your response, because I got quite a lot out of the post.
in a quest to avoid interacting with the layer of intentions
Like, I think this post has a true description of a key part of what's going on. The key insight is that your working memory is limited to a few slots, and that if you have more you'll be able to see a few more levels of modelling of modelling of modelling etc, and I think the descriptions are accurate portrayals of causally what happened. I think that, especially in the modern tech era, a lot of norm violations come down to having very different assumptions about background context leading to mixed signals, and having common norms for carefully and slowly making a lot of the background assumptions explicit, can lead to resolving problems that otherwise would be intractable or be resolved with a lot more violent force. You can't say "Make everything explicit", but this post helps set out a framework for making certain important things explicit.
I agree that there are other skills that need to be done here, but this to me feels pretty key (and not just like a few bells and whistles that are distracting from the real substance of what needs to happen).
Am curious to know more of your thoughts.
I think the post imagines something like a multi person stack trace. In reality backwards facing introspection winds up confabulating, and there's not limit to how many epicycles can be added with multiple parties confabulating.
Hmm. It seems like a pretty strong claim to me that the backwards facing stack-trace is purely confabulatory, or is so confabulatory that it's not useful to understand.
I do agree confabulation can happen (moreover, I agree that confabulation is happening all the time). But my current sense is that the confabulation is... happening in a fashion that is entangled enough with reality that it usually collapses into something real-enough to be worth talking about (both in multi-person relationships and other domains).
I do find it somewhat plausible that the overall approach and stance in this article isn't the based way to handle social conflict, but that doesn't seem to be because the content is false, so much as it's an unhelpful frame. (I'm currently somewhat agnostic on how helpful the frame is, and think it varies depending on the situation)
It would help me to evaluate this post if people who have attempted this technique would report on how it went, and whether or not they found that they reached more than three levels of meta.
Very interesting analysis, thank you! This unrolling technique seems to be quite useful, at least in some situations, where, as you said people are "generally open minded enough to be willing to try out a "weird conflict resolution technique I'm experimenting with"".
One comment I have is that this is not what I understand metacognition is like, thinking about thinking about thinking ... . This appears to me more like a linear chain of inferences than a stack of them. So, in a comp-sci terms, you go through the list, rather than unroll a stack. The difference is that to get to "Bailey observes (1.1), and feels it was bad," one does not have to pop every single level from the stack, but can actually go through the chain of inferences back and forth from any element. Or at least that is what it seems like to me.
As for your point that people shy away from posting "mediocrely-explained ideas", I agree, especially if the ideas are not in the mainstream. *looks at the backlog of a dozen drafts* nope, totally not what I am doing.
As for your point that people shy away from posting “mediocrely-explained ideas”, I agree, especially if the ideas are not in the mainstream.
This might not be a bad thing. Idea inoculation is a known concept in sociology, and it can make sense to not post an explanation of something if you think that it will cause people who have heard your explanation to discount future versions of that idea.
To quote Nietzsche:
There are terrible people who, instead of solving a problem, bungle it and make it more difficult for all who come after. Whoever can't hit the nail on the head should, please, not hit at all.
See this paper arguing that humans have the ability to solve tasks that require up to seven levels of recursive metacognition, especially when those tasks are ecologically valid (i.e. the prompts are films of interactions, and the question is to pick which sentence one of the participants is more likely to say). Abstract:
Recursive mindreading is the ability to embed mental representations inside other mental representations e.g. to hold beliefs about beliefs about beliefs. An advanced ability to entertain recursively embedded mental states is consistent with evolutionary perspectives that emphasise the importance of sociality and social cognition in human evolution: high levels of recursive mindreading are argued to be involved in several distinctive human behaviours and institutions, such as communication, religion, and story-telling. However, despite a wealth of research on first-level mindreading under the term Theory of Mind, the human ability for recursive mindreading is relatively understudied, and existing research on the topic has significant methodological flaws. Here we show experimentally that human recursive mindreading abilities are far more advanced than has previously been shown. Specifically, we show that humans are able to mindread to at least seven levels of embedding, both explicitly, through linguistic description, and implicitly, through observing social interactions. However, our data suggest that mindreading may be easier when stimuli are presented implicitly rather than explicitly. We argue that advanced mindreading abilities are to be expected in an extremely social species such as our own, where the ability to reason about others' mental states is an essential, ubiquitous and adaptive component of everyday life.
[ETA: the paper was actually brought to my attention by the author of the OP]
I've thought a lot about this post in the last year, and also referenced it a few times in the broader context of talking to people about ideas around common-knowledge. I think it, together with Ben's post on common knowledge communicates the core concept quite well.
I think the key point is that situations where A thinks that B thinks that C thinks that X don't require both A and B to be purely guessing. Instead, C might have told B they think X, and A might have observed that conversation. It feels to me that when people point at the difficulty with high levels of meta, they mean the difficulty of guessing through multiple levels. The examples in this post involve just observing what's going on, which shows how it's often easy to form such states of knowledge in practice, if you don't have to guess.
I don't particularly agree about relevance of the chunking words, as the examples are only specific models of their meaning, the meaning itself can't be unrolled into a deep explicit chain of meta, it's more of a forest of possibilities than a chain, which is unlike the usual setting of "many levels of meta". It does turn into more of a chain when there's enough data, but then the words themselves become only a small part of the relevant state of knowledge.
1.4) Alex reflects on feeling judged, doesn’t like it, and concludes that Bailey is “a downer”.
Unrolling of referents: Alex felt it was bad that Alex felt that Bailey felt that Alex leaving out the milk was bad.
There seems to be some ambiguity here, which I found very confusing on a first read-through, and which I’m still not sure I’m interpreting in a way that makes any sense.
Do you mean:
“Alex felt bad, due to the fact that Alex felt that Bailey felt …”
Or:
“Alex felt it was bad that Alex felt that Bailey felt …”
In other words, is it that Alex dislikes being made to feel judged? Or, is it that Alex views his own response (feeling judged) to be somehow un-virtuous?
Edit: Also, how/why does Alex conclude that Bailey is “a downer”? How does this follow from the rest of it? I don’t get this part.
Edit2: I also don’t get this part:
2.5) Charlie observes Alex treating Bailey like “a downer”, thinks this is baseless, and feels Alex is “a snob”.
A snob? What? I don’t understand how anything like the usual meanings of the concept of “being a snob” can make this reaction make sense.
Something strange is going on with your word/concept usage here, and I’m more and more perplexed about what it might be.
In other words, is it that Alex dislikes being made to feel judged? Or, is it that Alex views his own response (feeling judged) to be somehow un-virtuous?
Pretty sure it's just that Alex dislikes feeling judged, rather than an evaluation of whether his own response was virtuous.
1.4) Alex reflects on feeling judged, doesn't like it, and concludes that Bailey is "a downer".
Unrolling of referents: Alex felt it was bad that Alex felt that Bailey felt that Alex leaving out the milk was bad.
2.4) Multiple similar Scale 1 events happen Alex does something X, and ends up feeling that Bailey was "a downer" about it.
Complete unrolling of referents: Alex felt it was bad that Alex felt that Bailey felt that Alex doing X was bad, for multiple values of X.
In the first example, there's [Alex feels that Baily [feels that leaving out the milk is bad]]
Then Alex feels that that^ is bad, which probably isn't 'I feel bad about having this understanding of the situation' but more like 'I disapprove of this thing that I understand'. So Alex is grumpy or angry or frustrated or disappointed when looking at [their belief that Bailey thinks their actiosn are bad]
I'm guessing the second example follows the same pattern. There's [Alex has a general feel that Baily [has a general feel that Alex is doing bad stuff] where the word felt that Critch wrote is like a belief but isn't necessarily explicit or justified.
So then you get Alex felt bad about the fact that, from Alex's perspective, Bailey keeps ETC.
I think that's different from both of the options you had in your top comment, but it's closer to your due to the fact option. There's a thing happening, and Alex doesn't like it (is sad, irritated, mad, whatever, which Critch tagged as just 'bad')
For your first edit, the downer thing reads to me as being more like 'Baily is a bummer or a party pooper or likely to have emotional reactions to things that I Alex think are too small or too trivial to deserve such a reaction.'
For your second edit, the snob thing reads to me as 'Charlie sees Alex judging Bailey as a person over Baily's judgements of Alex, and furthermore this pattern-matches for Charlie to a kind of snobbishness, E.G. Alex is judging Baily for not living up to Alex's standards of what should or should not trigger emotion or what one should or should not do in response to emotion, and maybe Alex is being hypocritical too since they're being emotional in response to that.'
So then you get Alex felt bad about the fact that, from Alex's perspective, Bailey keeps ETC.
What does ETC mean here? I thought maybe you meant "etc." but I can't get the sentence to make sense in my head when I read it that way.
I meant etcetera.
Alex felt bad about the fact that, from Alex's perspective, Baily keeps having bad feels about actions that Alex is taking (things analagous to leaving out the milk)
In extrapolating the writing time on this post, the votes on this post (53) in a few hours since writing and the method itself...
I suspect you have communication abilities that are above the average human and there's more to this method than you are putting into words in the first instance.
For example, hearing the "yes" when someone said "no" to feeling judged.
That's great for you, but unfortunate for anyone replicating. Do you have any comments on this problem?
I also find the specifics of the method unclear. When he shared it in a lightning talk a few years ago, the point that humans model each other recursively like this was the useful part for me.
Thank you so much for writing this! I remember reading a tumblr post that explained the main point a while back and could never find it again -because tumblr is an unsearchable memory hole- and kept needing to link it to people who got stuck on taking Eliezer's joking one-liner seriously.
If you like this post but want more examples, Knots by R.D Laing is a book full of them. http://www.oikos.org/knotsen1.htm
I think this post crystallized a concept I had not previously had and will be useful for writing about rationality
This post (alongside Duncan's post Common Knowledge and Miasma) has been in my head since I read it. It has directed my attention to the many instances in which I can unroll a "simple" seeming judgment into multiple levels of modeling people's metas.
Things I'd like to see:
The complete unrolling of 2.5 (and thus 2.6) feel off if they are placed in the same chain of meta-reasoning. Specifically, Charlie doesn't seem like she's reacting to any chains at all, just the object-level aspect of Alex pegging Bailey as a downer. I can see how more layers of meta can arise in general, but in situations like these where a third person arrives after some events have already unfolded doesn't feel like it fits that model very well - is the claim that Charlie does a subconscious tree search for various values of X that might have caused such a chain of interactions, and then draws conclusions about the baselessness of the 'downer' brand based on that?
It seems that a large subset of issues in situations like these but perhaps more grave is that Bailey does indeed do 2.6 exactly as stated, except it's based on a non-existing chain in 2.5, leading to a quagmire of false understanding.
Specifically, Charlie doesn't seem like she's reacting to any chains at all, just the object-level aspect of Alex pegging Bailey as a downer.
I sort of agree. But in the cases where Charlie knows what happened, you might expect that their evaluation of whether Alex was right to conclude that Bailey is a downer might depend on the full chain of events.
This post seems to be helpful, and it is nice to see that meta-level reasoning is something that happens in ordinary situations, though the exact number of levels of meta reasoning isn't particularly clear.
A point of ambiguity here seems to be when you are supposed to use this as a conflict resolution strategy, though I suspect this may be due to an imperfect understanding of this as you present it. Mostly it's just that my intuition appears to say that this isn't a fully generalized system for resolving miscommunications given willingness to go along with it.
Side note, but I really appreciated the bolded sentences marking the start and end of the 'tiring symbolic reasoning' section.
I normally give up on posts on this sort of topic precisely because I can see that I'm getting into an unknown amount of unpleasant mental effort holding all the "he said she said she said"s in my head at once. This time I could quickly gauge how much of that stuff there was, and it looked manageable, so I persevered.
I once heard an academic talking on the radio about social metacognition. He pointed out that some episode in Shakespeare's Othello involves 7 levels of metacognition. 5 of them are in the play (unfortunately I can't recall the details, but someone thinks that someone else knew that someone else deceived someone into thinking that blah blah). The sixth is the fact that the audience understands it, and the seventh is that Shakespeare knew that by writing this he could make the audience understand it. (And arguably there's an eighth, that we realize that Shakespeare knew that...)
It only took the academic a few sentences for him to summarize the whole situation and all the levels, and it was all perfectly clear. Showing how good humans are at advanced metacognition.
Also, incidentally, metacognition is crucial in professional level poker, in which multiple levels are used - e.g. by betting this I can make my opponent think that I think that he thinks that I have an ace.
From the example it looks to me like the disagreement happens immediately, but I am not sure; I may be misreading this because of the ambiguity of the word "bad" in combination with the reality-tracking of the milk spoilage and Alex's feeling judged. What I would expect to see in unrolling the referents for each part is this:
1.1) Alex leaves out the milk for 5 minutes
1.2) Bailey observes (1.1), and feels it was bad.
Unrolling of referents: Bailey felt that Alex leaving out the milk was bad.
1.3) Alex observes (1.2), and feels judged.
Unrolling of referents: Alex felt that Bailey felt that Alex is a bad person for leaving out the milk.
Bolded the difference. This might have been within the compass of the intended meaning, but the value of unrolling the referents is not clear unless it reveals a disagreement somehow, and by my reading it doesn't appear that there is any disagreement in the initial example.
When you gave your real-life example, it appeared that you noticed a likely source of disagreement, and then used the unrolling trick to get the other party to notice that the disagreement was where you already thought it was. Have any of the cases where you used this method so far been ones of mutual bafflement?
I also agree that things should be written even if the idea may not be perfect, but that imperfection is what helps improve and become a clearer model. And, Bravo on at least the first polemic to kinda support your idea about the levels of metacognition.
AND OMG does this really happen? cuz i think now i can conjecture as to why i was a social pariah. Are feelings the primary mover for most homo sapiens? I thought it might be reason but i guess this is a very specific circumstance and the parties involved do prioritize feelings and assumptions before reason? When we go around acting motivated by feelings, judgings, and assessings how others view us doesnt seem to fit the criteria for distinct layers of metacognitive states. Perhaps if one were to ask why is there judgement in the first place and in the second do i care enough to act on it? At the very least ask, 'do u think that was bad?"
I dont even understand why all these feelings are being volleyed around when they dont have to be. Are these intrinsic cognitive acts or an extrinsic ones? Is this a first person layer or a third person layer or is it the layer of the author's metacognition?
I don't think that some of your feelings (the examples' feelings) count as different levels of metacognition. The first person would still run into a self reference recursion, e.g. the feelings of their feelings and thoughts on their thoughts ad inf.. The only person for whom perhaps more levels of metacognition exists is the third person omniscient. I think that these may prove to be factors when developing ideas about levels and layers of metacognition and perhaps as an actor in this scenario, ironically peel away and simplify the layers.
I hope that this heuristic is not the norm for our kind in society; The amount of hours wasted must be staggering.
Are feelings the primary mover for most homo sapiens?
Yes. In fact, there's some lines of research suggesting that feelings are the primary mover for all homo sapiens, which seems plausible to me; people who think of themselves as "not feeling-driven" often seem to just have a negative emotional reaction to their stereotype of feeling-driven decision-making, and a positive emotional reaction to what they think of as non-feeling-based decision-making.
Disclaimer: This post was written time-boxed to 2 hours because I think LessWrong can still understand and improve upon it; please don't judge me harshly for it.
Summary: I am generally dismayed that many people seem to think or assume that only three levels of social metacognition matter ("Alex knows that Bailey knows that Charlie knows X"), or otherwise seem generally averse to unrolling those levels. This post is intended to point out (1) how the higher levels systematically get distilled and chunked into smaller working memory elements through social learning, which leads to emotional tracking of phenomena at 6 levels of meta and higher, and (2) what I think this means about how to approach conflict resolution.
Epistemic status: don't take my word for it; conceptual points intended to be fairly self evident upon reflection; actual techniques not backed up by systematic empirical research and might not generalize to other humans; all content very much validated by my personal experiences with talking to people about feelings in real life.
Related Reading: Duncan Sabien on Common knowledge & Miasma; Ben Pace on The Costly Coordination Mechanism of Common Knowledge
I. Conceptual introduction, by example
Here's how higher levels of social metacognition get distilled down and represented in emotions that end up tracking them (if poorly). Each feeling in the example below will be followed by an unrolling of the actual event or events it is implicitly tracking or referring to.
Warning: reading this first section (I) will require a fair bit of symbolic reasoning/thinking, so you might find it tiring and prefer to skip to later sections. A better writing of this section would do more work in between these symbolic reasoning bits to distill things out and make them easier to digest.
Scale 1: One event, four levels of meta (yes, we're starting with four)
1.1) Alex leaves out the milk for 5 minutes
1.2) Bailey observes (1.1), and feels it was bad.
Unrolling of referents: Bailey felt that Alex leaving out the milk was bad.
1.3) Alex observes (1.2), and feels judged.
Unrolling of referents: Alex felt that Bailey felt that Alex leaving out the milk was bad.
1.4) Alex reflects on feeling judged, doesn't like it, and concludes that Bailey is "a downer".
Unrolling of referents: Alex felt it was bad that Alex felt that Bailey felt that Alex leaving out the milk was bad.
Notice that the unrollings look and sound very different from the distillations. That's in large part because the unrolling is not our native format for storing social metacognition; it's stored via concepts like "feeling judged" or "being a downer". However, to the extent that the feeling "Bailey is a downer" is tracking something in reality, it's tracking things that track things that track things that track reality: in this case, milk spoilage.
(An aside: notice also that 1.4 involves Alex's feelings about Alex's feelings. Some people wouldn't call that an extra level of social metacognition, and would just combine it all together into "Alex's feelings". However, I'm separating those layers for two reasons: (1) the separation in counting won't affect my conclusion that the total number of levels being implicitly tracked greatly exceeds three, and (2) I think it's especially important to note when people have feelings about their own feelings, as that can lead to circular definitions in what their feelings are tracking; but that's a topic for another day.)
Scale 2: multiple events, six levels of meta
I'll start the numbering at 4 here:
2.4) Multiple similar Scale 1 events happen where Alex does something X, and ends up feeling that Bailey was "a downer" about it.
Partial unrolling of referents: Alex feels that Bailey is often a downer
Complete unrolling of referents: Alex felt it was bad that Alex felt that Bailey felt that Alex doing X was bad, for multiple values of X.
2.5) Charlie observes Alex treating Bailey like "a downer", thinks this is baseless, and feels Alex is "a snob".
Partial unrolling of referents: Charlie felt it was bad that Alex felt that Bailey is a downer.
2.6) Bailey observes Charlie's opposition to Alex's snobbiness, feels socially included by Charlie, and concludes that Charlie is "protective & welcoming."
Partial unrolling of referents: Bailey felt it was good that Charlie felt it was bad that Alex was feeling that Bailey was often a downer.
Complete unrolling of referents: Bailey felt it was good that Charlie felt it was bad that Alex felt it was bad that Alex felt that Bailey felt it was bad that Alex did X, for multiple values of X.
If you count the (+/-) signs implicit in the "good" and "bad" judgements here, they suggest that Charlie is implicitly condemning Alex's multiple "X" behaviors (perhaps among other things, such as Alex's style social of delivery). Charlie might not intend or even be aware of this effect, and it might take explicit work and discourse for the group to untangle and notice. What could result from the group reflecting upon this together? Well, it's not hard at all to imagine that Charlie, upon haring about the milk and other situations (values of "X"), might conclude that Alex's behavior with the milk was reasonable, and that "Alex was right to think that Bailey was being a downer". In turn, this could burst Bailey's bubble of social support, and result in Bailey having a change of heart about how critical to be of others.
The particular consequences here are of course hypothetical, and things could go very differently depending on the details; it's just meant to illustrate how "changes of heart" can propagate through unrollings of social metacognition.
ETA (8/27): Note there is an important distinction here between implicit and explicit metacognition. Bailey alone is not (necessarily) loading up a 5-layer-deep cognitive model of what's going on, all at once. Rather, the layers are distributed across people, whence the term "social metacognition". However, six levels of metacognition really are needed for someone to aiming to ground out all these feelings in "object-level" reality (i.e., non-mental phenomena like milk spoilage).
(This is the end of the tiring symbolic reasoning section.)
II. Higher levels of meta
Without going into further explicit detail, I hope you can see the pattern. Levels of social metacognition get distilled into simple, repeated concepts like "feeling judged", "being a downer", "being a snob", "being welcoming", and so on. To the extent that these distilled concepts behave in a somewhat systematic manner in relation to reality, they have some tendency to be actually tracking things that are actually happening. It's not uncommon for me to observe six levels of social metacognition in a given disagreement or conflict, which is why I chose six for this post.
III. People don't usually unroll things this way. Why?
Unfortunately, I think a lot of people aren't aware that it even makes sense to try to ground out these sorts of social metacognition in more explicit terms to be reasoned and disagreed about. I think this is because it takes a lot of working memory slots to do, such that you basically need a shared piece of paper, whiteboard, or a shared Google doc to do it reasonably and collaboratively (rather than just slinging hard-to-unpack negative judgments at each other, adversarially, either in person or on the internet). However, I've resolved conflicts through co-writing and co-diagramming relevant levels of social metacognition many times now, and found it to be very enlightening almost every time it a way that directly benefitted the "social situation". I've found it's best if the shared writing medium is used for distillation mechanism, and is augmented by actual real-time conversation over the creation of the document.
IV. A fruitful application
For instance, in the past week, Alex (anonymized) felt judged by me for a thing I noticed Alex doing. I said, (a) "Don't worry, I don't think you did anything bad", but Alex didn't find this reassuring. To check, I asked "Do you feel like I feel like you did something bad?" and Alex said "No". This ran up against my explicit models of people feeling judged that had fit well with past unrolling of the concept. So, I broke out a Google doc (in person) and started unrolling stuff. The situation was more complicated than described above, so the doc gave us mental space to explore other ideas for resolution. We eventually looped back to my question (a) above, and Alex said "Huh, yeah, I think I do feel that you feel that I did something bad." Once that awareness existed, I responded "Cool! Well guess what? I don't think you did anything bad.", and this time, it resonated with Alex and Alex no longer felt judged. I then apologized with "Also, I'm sorry you felt judged. Given that I didn't actually feel you were doing something bad, this was a mistake on my part, and I'm sorry."
This further cleared things up.
This whole process took about 15 minutes. In retrospect it might seem like we could have jumped straight to this solution by me saying "I'm sorry I made you feel judged", but that wasn't an available strategy ex ante, for two reasons:
(1) Sometimes I really am judging someone, and I'm okay with them feeling judged, because I do in fact think they did something wrong. As a result of this willingness in myself and others, it's not always believable to say "Sorry, I wish I hadn't made you feel judged". Indeed, to many this feels like a platitude. But, by actually going through the work of actually unrolling whether or not I thought Alex did a bad thing, and the other details of what was going on between us, we established enough shared clarity about the situation that we managed to "get on the same page" what whether a bad thing was done, who thought or didn't think that, and who miscommunicated or didn't miscommunicate about it.
(2) There were many other things going on that the Google doc helped to organize and sift through without getting us lost. Without that functionality, I don't think we would have been able to hone in on the particular narrative resolution above.
V. How generalizable is this 'unrolling' technique?
The application (IV) above is not an isolated incident. I've founding co-writing and co-drawing to be extremely valuable in settling social disagreements and conflicts on at least 30 occasions now, with at least 7 different people, of varying degrees of inclination toward explicit symbolic reasoning. I imagine some inclination is necessary, but much less than I would have expected previously. For instance, I've used this sort of unrolling heuristic fruitfully in numerous conversations with folks close to me who
(1) didn't go to college or otherwise study a symbolic discipline like math or linguistics, but who
(2) were generally open-minded enough to be willing to try out a "weird conflict resolution technique I'm experimenting with" where we sat down together and tried unpacked our feelings in explicit terms in a common medium (usually a Google doc).
I'll defer to the finding of the broader community here to see if others can make this sort of thing work usefully.
VI. Relation to "miasma" and "hype"
The concept of "miasma" that Duncan is gesturing at in Common Knowledge and Miasma feels like a real social phenomenon to me, succinctly definable as "negative ungrounded social metacognition". There is such a thing as positive ungrounded social metacognition, as well, which I think is normally called "hype", at least in Silicon Valley. I think both hype and miasma are failures of group coordination, and both are costly to resolve, along the lines pointed out by Ben in The Costly Coordination Mechanism of Common Knowledge. However, the communicative costs of resolving these problems can be significantly decreased if people are aware of what they are. Both require creating and sharing of ideas in places of common view, like writing blog posts that a lot of people see each other commenting on, or holding meetings that a lot of people can see each other attending, or for complex topics, sitting together and co-authoring a document.
VII. Apology
I'm sorry I put very little effort into the pedagogy of this post, due to having too little time to write it. Hopefully it will be of some value anyway, due to much better posts having been written and circulated on common knowledge recently, and due to the general intellectual health of LessWrong appearing, to me, to be able to absorb mediocrely-explained ideas and flesh them out into better ones. My sense is that the culture here has been trying to move towards people not waiting until an idea is perfectly elaborated before starting to talk about it, so to the extent these ideas might be valuable, I'm punting to the community to do more elaboration and/or distillation of them. Indeed, wishing not to be a part of a "common knowledge breakdown" problem is one reason I time-boxed two hours to write this post instead of waiting to improve it.