The feeling of breaking an Overton window

by AnnaSalamon1 min read17th Feb 202125 comments

103

Rationality
Frontpage

Epistemic status: real but incomplete observations.

In late February of 2020, I went to the grocery store at 2am with my husband (emptiest time), and we bought ~$1k of mostly canned or dry goods. We were the only customers in line. The cashier seemed interested in our purchases, and I felt myself stiffening as she looked. Then she asked me: “are you worried about that virus?”

And… I found myself reaching for a lie, trying to compose a lie, moving my speech-planning-bits as though the thing to do was to lie. I mean, not a technical lie. But I found myself looking for a way to camouflage or downplay my model of the virus.

Oddly, it felt more like a thing happening in me (“I found myself") than like a chosen thing. If you’ll pardon the analogy, it somehow felt at least a bit like throwing up, in that I remember once when I was trying not to throw up, and all of a sudden it was like an alien process took over my consciousness and throat, reached for the bucket, got my hair out of the way, and did the actual throwing up. And then returned my body to me when it was over. The "alien process" sensation felt a bit similar.

With the cashier, I wondered at my impulse as it was happening, but I couldn’t tell if the impulse’s source was “for the cashier's sake” (didn’t seem to make sense); “to prevent her from harming me” (didn’t seem to make sense); or … what exactly?

I forced myself to say “yes” to the cashier's question, and to elaborate a bit; to my surprise, she seemed sincerely curious, and told me several people had been in doing this and she would probably also prepare in some way. Even after this, my sentences wanted to (sound soothing? fit in? avoid disrupting others’ “normal”? I’m still not sure what), and it took me active effort to partially not do this.

I am somehow quite interested in what precisely was happening there, and in any related processes.

My guesses as to how to help with this puzzle-set, if you're so inclined:

  • Share observations (not theories) of any related-seeming things you’ve noticed (the rawer the better);
  • Share observations (not theories) of what it’s like to be you right now trying to look at this stuff. Do you have introspective access? Do you have sort of have introspective access, and in what way? Do you kind-of-like identify with it? Kind-of-like not-identify with it?
  • And okay, yes, also theories, I just hope the theory doesn’t overwhelm the observations at this confused stage.

103

24 comments, sorted by Highlighting new comments since Today at 5:09 AM
New Comment

If I reply with the naive factual response, “Yes I’m stocking up to prep for the virus”, and leave it at that, there’s a palpable awkwardness because all participants and witnesses in the conversation are at some level aware that this carries the subtext, “Yes I’m smartly taking action to protect myself from a big threat while you are ignorantly exposing yourself to danger”, which means a listener has to wonder if they’re stupid or I’m crazy. Even if the listener is curious and doesn’t take any offense to the conversation, they know that I’ve made a social error in steering the conversation to this awkward state, because it’s mutual knowledge that a savvy conversationalist needs to be aware of the first-order subtext of the naive factual response. The objective social tactlessness of my naive response provides valid evidence to update them toward me being the crazy one.

I think a more tactful response is, “Yeah, I know a lot of people say it’s not a big deal and I hope they’re right, but I think there’s enough risk that extra supplies might come in handy”.

If I first acknowledge and validate or “pace” the background beliefs of mainstream society, then it’s socially graceful to segue to answering with my honest beliefs. Now I’ve portrayed myself as an empathetic character, where any listener can follow my reasoning and see that it’s potentially valid, even if it doesn’t identically match theirs.

More generally, “portray yourself as an empathetic character” is a social skill I find myself using often. Basically copy the way the protagonists talk on This American Life, where even the ones who’ve done crazy things tell you their side of the story in such a way that you think “sure, I guess I can relate to that”.

Sometimes when asked a threatening question, or an nonthreatening question in a threatening situation, I get a sense of total blankness and loss of memory. I have no ability to remember or grasp at content relating to the answer to their question. I can see this from the outside, and still probably talk about other things, or if I wish, talk around the blank spot and hope to spiral into it. It feels like the piece of me with relevant knowledge is 'playing dead' until the threat is gone.

My natural inclination is to hold still and watch myself, and see the parts of me that feel alive slowly expand until they include the 'playing dead' part, and it wakes up, and then it might be able to speak. This is somewhat socially awkward. What happens in a conversation going at normal speed without room for a pause is that I feel a strong pull to reach for something other than myself that can speak for me. Anything but my words. Sometimes it's something i can look up, like a book, or document, (that maybe I can pull from my phone or memory), if not I feel a pull towards confirming the question asker's expectations and worldview. Like I can imagine that if I was in a similar situation of buying tons of food and they asked me if it was for the virus, but actually I was buying it for my nuclear bomb shelter, I would feel really easy to say "sure... yeah". Then having said something that was at least slightly in the direction of the truth, I would feel more free to follow up with "well actually...". But sometimes I just end up saying the non-confrontational false thing and it never gets corrected.

Usually this only happens when there are other social norms and pressures going on that make it so I feel unable to talk about or mention the sense of threat.

Sometimes unconscious/visceral fears are kind of what I thought they were:

If I look down from a tall height, I experience: [vertigo, a slight increase in my heart rate, some similar change in my breathing, a slight increase in my tendency to "freeze" my muscles, a shift in my attention toward the height/downness, etc.].

I might've thought this response was a consciously chosen strategy for not falling. Except that it also occurs when I'm walking on a glass floor in a well-engineered building I trust. Still, I conceptualize it as something like "in-built fear of heights; designed to prevent falling but based partly on a bunch of visceral cues that persist even when my conscious mind knows I won't fall". My lead "rationalization" for the response ("I'm breathing more shallowly and refusing to walk to the edge of the cliff with you because I don't want to fall off the cliff") is at least partly post-hoc, and is causally downstream of a more visceral reaction... that is basically also evolved for not falling off cliffs.

And sometimes unconscious/visceral reactions are just not at all what I thought they were:

Some years ago, I was waiting impatiently for a friend to leave an event we'd both been at, so that I could "go home to do my important work with its urgent deadline." Then I took a better look at the feeling, and realized that I was actually cold and in need of a washroom. I found a washroom and some warmth, and felt suddenly at ease and untroubled about my task. I've related differently to such feelings ever since.

Similarly, some years ago I was hanging out with my cousins around a holiday, and I felt "bored" (as I saw it) "because" they weren't talking about anything interesting, and I couldn't share with them my thoughts about anything interesting, such as AI risk. And then we switched board games, and I started laughing more, and I realized that my previous "bored" has actually been made of "socially uncomfortable". And I got a bit better at identifying the "socially uncomfortable" visceral response, and calling it by that name instead of some other.

The interesting thing for me about the cashier incident above, was the rationalizations my brain produced around the "don't share my views" impulse in my cashier situation seemed pretty transparently unsound (in my situation, in contrast to e.g. Oliver's similar instance in-thread, or any number of previous times when I had more plausible ... rationalizations? reasons? for not breaking an Overton window). If I was concerned about the cashier's welfare: yes, she might find it uncomfortable to see our views, but "withholding practically relevant information from someone specifically asking about it" does not seem kind. If I was concerned about causing some sort of social drama that might do me harm: it didn't seem like there was much plausible harm. (The risk that she might object to selling me the groceries did not occur to me, I think correctly. The 24-hour Safeway just outside of Berkeley may have had different customers than the downtown Berkeley Trader Joe's that Oliver and Ben Pace mention. She seemed peaceable/stable/normal. I was with my husband, which is probably safer than alone. Etc.) There probably was still more social drama in telling the truth vs in evading, but its absolute amount seemed pretty small, to the point where the normal ratios at which I try to buy [human decency / communication / being helpful and honest] vs [avoiding harm to myself] seemed to pretty clearly favor talking.

So it was more a glass floor situation, than an actually being near a cliff situation. Useful for elucidating things. And I'm not yet sure what the analog of "avoid falling" is, that this reaction is actually triggered by cues of. Is there a pretty in-built visceral thing for "don't break Overton windows", that is quasi-independent of conscious knowledge that you're safe? Is its true name "don't break Overton windows", or something else? What's up with the way the impulse in me oscilated between selfish rationalizations ("she might harm me") and morality-related rationalizations ("it's wrong to upset people")?

And I'm not yet sure what the analog of "avoid falling" is, that this reaction is actually triggered by cues of

"Falling" might be "acknowledging a truth which someone really wants to keep hidden"? Some related examples:

  • A totalitarian ruler is trying to maintain a narrative that their country is prospering, by punishing anyone who calls attention to problems
  • A manager worried about legal liability for a problem, who is trying to avoid the creation of evidence that they knew about the problem
  • A criminal organization, which punishes anyone who draws attention to its crimes
  • Someone who would be obligated to do difficult things about the problem if it were acknowledged, who socially attacks anyone talking about the problem

My not-well-grounded impression is that these sorts of situations are common, are difficult to rule out without false negatives, and are at the heart of Maziness.

I'm a teacher and in the country where I live we had a few weeks of school in-person this September-October until the fall wave got too high and we all went home.  School was run with a reduced class size and distancing rules in place, but try telling that to a sixth grader.  At one point a kid jumped out of his chair and ran up to me and I had to tell him to back up and go back to his seat.  Another kid asked, "Mr. Z, are you scared of coronavirus?"

What did that feel like?  It felt like being offended.  I sort of mentally seized up, and my chest muscles clenched.  I was able, with some effort, to acknowledge that yes, coronavirus was a concern for me.  I've thought about that moment a lot.

My theory at the time was that we're conditioned to avoid admitting to being afraid of something, especially publicly and on short notice.  At least when I was growing up, if I admitted fear, I'd generally be mocked for it, so I learned not to do it.  And when someone asks me if I'm afraid - or accuses me of being afraid - I get angry at them.  I want to tell them, "no, I'm not 'afraid' - I'm rationally concerned about bad outcomes."  I think in part the accusation of fear angers me because it implies that I am somehow being irrational, or letting my emotions get the best of me - whereas my belief is that if I'm "afraid" that means that there is real danger which is dangerous and likely enough to warrant some action to mitigate the danger.  But in general, if someone asks me if I'm afraid, what it feels like is they're accusing me of being irrational.  So my impulse is to deny it, or to try to "rationalize" the fear.

But I think this is a destructive impulse - I think being afraid to admit you're afraid leads people towards collective stupidity, like reopening schools before it is safe to do so.  Speaking of which...

This Monday we're supposed to return to in-person instruction, despite cases being almost as high now as they were when we closed.  Parents are just tired of their kids, I guess.  Anyway, because I have a specific risk factor, and live in a multi-generational household, I asked my principal if I could continue teaching online and just have them project me into the classroom for the seven kids at a time that will be allowed in the room in person.  I ended up speaking with the director of the school yesterday and she told me I sounded scared.  Without hesitation I agreed that I am indeed scared.  But being able to smoothly admit that seems to have taken, let's say, many hours over many months of introspection and reflection and mental preparation and specific resolve.  Basically I had to have so much confidence that my fear was justified and that mitigating the danger was necessary that I felt like I could absorb the hit to my social status or reputation that would come from being a person who admits to being afraid.  Or at least, that's what it looks like from the inside.  I know that I did the introspection and preparation, and I know that my physiological and mental reaction to being asked if I was afraid of coronavirus changed between October and February, but I obviously can't prove that the introspection and preparation caused the change.

Social acceptability , that is what all interaction are made off.

 

At the beggining of the pamdemic I wore a mask for non pandemic reason and a person made a joke to me laughing "Is that for the corona virus ?" , months everyone wore them for pandemic reason , and no one made a joke about it (and probebly didn't rembmer their previous attitude).

I experienced almost the exact same thing in almost the exact same situation. This was mid-February and we were getting ready to lock down and were buying about $500 worth of beans and pasta from the Trader Joe's nearby. The cashier asked whether we were worried about the virus, and I felt this really strong urge to lie or make up something that didn't sound concerning. 

I do think this was not completely unjustified. A day earlier we tried to buy food from Costco, but a lot of the staples were gone, and all the cashiers seemed very angry with the "preppers" who just bought all the shelf-stable food, and it didn't seem impossible this cashier would react similarly. 

In the end, I evaded the question, saying something technically true without speaking about the issue. I think I said we were in a large group house and were cooking food for everyone, which was true, but also didn't really respond to the question. 

I mentally came back to that scenario multiple times over the pandemic and also thought a lot about why I was so hesitant to say anything about my concerns. I definitely have a sense that you are not supposed to have "real" conversations with cashiers, because those would take up time and get in the way of other people, but I had a sense that bringing up such a concerning hypothesis in such a time limited context is bad. I also felt just object-level afraid that they might refuse to allow us to actually buy the food if I gave them a straightforward answer.

Habryka and I went together to buy food from the Trader Joe's. I don't recall what was or wasn't said.  All I recall is that I was concerned that they wouldn't let us buy so much food, that some staff member would come out and get angry at us and stop us, like I was at school and breaking the rules. But they were notably positive about it ("Wow! This is a fun order!"), and I felt relief and was extra friendly.

Added: We went at like 6pm I think? There was a lot of people around, and I felt a bit conspicuous. I acted normal. I think I was thinking about how to get around the system if they stopped us. I think I had some vague fantasy of coming in and out a lot in different wigs and noses, which is not a very realistic plan, but I was sure concerned that I wasn't allowed to buy so much, and was sort of mentally preparing for that world.

At least for me, I think the question of whether I'm buying too much for myself in a situation of limited supplies was more important for the decision than the fear of being perceived as weird. This depends of course on how limited the supplies actually were at the time of buying but I think it is generally important to distinguish between the shame because one might profit at the expense of others, and the "pure" weirdness of the action.

Totally. I was not AFAICT worried at the time about limited supply buying, or not very worried; the Safeway we were getting things from did not seem out of much and I hadn't heard people complain about shortages/buying yet as far as I can recall.

I can share observations/thoughts about some similar experiences.

Sometimes I lie.

But let me back up: for a long time, I've been aware of two different modes in which I speak -- one is fluent and "real-time"; the other is slow, halting, and feels somewhat like wearing mittens, somewhat like trying to squash high-dimensional objects down into a lower-dimensional space, somewhat like.. tasting alternatives for the right connotations and inflections. Sometimes this second mode is evoked when some part of me becomes concerned about doing PR with the person I'm talking to, or if its a topic where I care about nuance or precision. I'd estimate that ~80% of my speech is in the fluent mode.

A few years ago (a few years into my meditation practice) I realized that speech in the fluent mode is, in some, way non-conscious. Sometimes I could become aware of speech just... unspooling through me, without intervention from... "me". The part of me that calls itself "me". The "global work space" of consciousness. Sometimes it was sophisticated speech! This has never happened in mittens-mode. Needing to sieve up a new/novel/unusual semantic meaning from the depths of language-space seems to require... "me".

Once, fluent-mode upset someone and I tried to explain this to them. It... wasn't well-received.

More than 99.99% of the time, I'm entirely happy with what, fluent mode utters. Very rarely, though (probably on the order of once every year or two) it will just lie. "How do you know X?" "Oh, through my dance community" <I actually met them through OkCupid>.

This tends to happen when I sense that the other person will get a negative emotional charge out of the truth, and/or if I have some negative emotional content around it. @AnnaSalamon, you gave me the model I use for thinking about this years ago: the parent who on picking up the phone angrily asks "Why don't you call more often?!" and then wonders why their child doesn't call more often. If a person has shown me systematically in the past that they will have a strong negative emotional reaction to the truth, then I powerfully learn their revealed preference that in some situations they hate the truth. I also learn an upsetting reciprocal lesson about myself and find myself in an awkward to back out of situation.

These moments are particularly vexing because they're extremely difficult to "train against" -- I have to be caught unaware, and in the right mental state for this to happen.

(Conversely, sometimes the babble system which Scott dubs the guf isn't generative enough in real-time. There have been various babble exercises posted here in the past year; other ideas I've had include learning to rap (badly! Badly! I don't expect to learn to rap well!) and trying comedy or theatre improv (Keith Johnstone's Impro seemed to be recommended a lot in rationalist circles a few years ago). One thing I wonder about: will training the guf to be more generative increase the risk of the guf make utterances I wouldn't fully endorse?)
 

I've noticed a thing whereby sufficiently out of the norm behavior can feel accusatory of others' risk models along with a corresponding urge to paper over the friction that sounds similar to what is described here.

[I felt inclined to look for observations of this thing outside of the context of the pandemic.]

Some observations: 

I experience this process (either in full or the initial stages of it) for example when asked about my work (as it relates to EA, x-risks, AI safety, rationality and the like), or when sharing ~unconventional plan (e.g. "I'll just spend the next few months thinking about this") when talking to e.g. old friends from when I was growing up, people in the public sphere like a dentist, physiotherapist etc. This used to be also somewhat the case with my family but I've made some conscious (and successful) effort to reduce it.

My default reaction is [exagerating a bit for the purpose of pulling out the main contures] to sort of duck; my brain kicks into a process that feels like "oh we need to fabricate a lie now, focus" (though, lying is a bit misleading - it's more like "what are the fewest, least reveiling words I can say about this that will still be taken as a 'sufficient' answer"); my thinking feels restrained, quite the opposite of being able to think freely, clearly and calmly; often there is an experience (reminiscent) of something like shame ; also some feeling of helplessness, "I can't explain myself" or "they won't understand me"; sometimes the question feels a bit intrusive, like if they wanted to come in(to my mind?) and break things. (?)

Some reflections:

  • "Inferential distance and the cost of explaining":
    • This is very viscerally salient to me in the moment when the "alien process" kicks in. I basically have the thought pattern of "They won't understand what I'm talking about. Mhh I guess I could explain it to them? But that will be lengthy and effortful, and I don't want to spend that effort."
      • I think this pragmatic consideration is often legitimate. At the same time I also suspect that my mind often uses this as an excuse/cover-up for something else.
      • For example, I am on average much less reluctant to give answers to such questions in English compared to German or French. I think about my work in English, thus, explaining my beliefs in another language is extra costly because it requires lots of non-trivial translation. That said, speaking German or French is also correlated with being in specific environments, notably environments I grew up in and that trigger memories of older self-conceptions of mine, and where I generally feel more expectations from the society, or soemthing.
  • "Updating based on someone's conclusions (including their observed behaviour) is often misleading (as opposed to updating on based one someone's reasoning/map)":
    • Based on the above, if I inner sim telling someone about my belief X that is, say, slightly outside of their overton window , I feel kinda doomy, like things will go wrong or at the very least it won't be useful. So, it feels like I either want to get the chance to sit down with them for 2h+ or say as little as possible about my belief X.
    • I think it's interesting to double click on what "things go wrong" means here. The two main things that come up are:
      • An epsitemic worry: they will objectively-speaking make a wrong update and walk away with more wrong rather than less wrong beliefs
      • A ~social worry: all they will update about is me being ~weird. A decent part of the worry here is something like: they will distance themselves from me because they will feel like they can't talk to me/like we're not talking the same language, a sense of isolation. Another part seems more extrem: They will think I'm crazy(?) (I sort of crinche at this one. I don't really think they will think I'm crazy(?). Idk - there is something here, but I'm confused about what it is.)

I didn't have any similar experience when I went covid-prep shopping; I was happy to tell people my reasoning, iirc. I think maybe one of the roommates I was with expressed some concern that I was spreading panic, and should avoid explaining myself. But my memory is fuzzy on that point.

I do remember that one cashier gave some serious pushback, saying people were "running around like chickens with their heads cut off" (or some similar expression) all due to "a cold virus" (or some similar phrase). The person behind us in line (not a roommate or acquaintance) backed us up, saying something like "that aint no cold virus, people are dying".

Later, an ex of mine who I occasionally keep up with was like "so you're one of those preppers who ruined things for everyone else by buying all the toilet paper". I experienced some indignation and a desire to defend my strategy as the one everyone should have followed.

A friend of mine reported that he discovered he was not nearly so ready to violate norms as he expected: suiting up to buy groceries felt viscerally wrong in a way that threatened his self-image as someone who readily violated norms. He formed the impression that really, he had just fallen into a role where he was the guy who "didn't fall into rolls", playing the part at parties, etc.

I live with someone who has anxiety.

This puts a strong damper on my ability to communicate with them in terms of likelihood or confidence levels. I don't have any expertise in the matter, but from the outside it sure looks like anxiety makes people hallucinate the I am under threat feeling, and that these hallucinations always appear whenever there is uncertainty.

This causes me to feel a similar impulse for dissembling whenever questions about how likely something is or what the might happen come up. Sort of like an artificial closing of the Overton Window.

The solution we have evolved is that I project strong confidence about the right thing to do (based on my internal estimates of the uncertainty), and they try not to think about what it is exactly that makes me so confident.

Share observations (not theories) of what it’s like to be you right now trying to look at this stuff.

Thinking about this situation, I feel that same sense of awkwardness and discomfort at the idea of saying something like, "Yes, I'm stocking up with a ton of food because I'm worried about the virus." I also get a sense of that same alien process driving me away from such an answer.

I think this post has a lot to do with shifting Overton's windows:

 https://www.lesswrong.com/posts/HLqWn5LASfhhArZ7w/expecting-short-inferential-distances

Preventing autism induced ire from others throught masking sounds somewhat similar. Sometimes you want to mask tot he extent that you can but you just can't all the way. Expecting people to be weirded out but the "deficiency" not being an internal signal to change stance.

Depersonalization when trying forcefully to fit illfitting social roles is also not unheard of.