Decision Theory: Newcomb's Problem


Covid: CDC Issues New Guidance on Opening Schools

In terms of concrete measures for lower-covid-risk school reopening, I think it's worth underlining massive use of HEPA filters. They are not that expensive, and should reduce aerosol density and thereby covid risk rather a lot. Worth trumpeting also for grocery stores, pharmacies, offices, etc.

I like your overall point more though, and feel a bit innane commenting about something else without responding to that.

Covid: CDC Issues New Guidance on Opening Schools

The big thing is that a clear majority thinks schools should not wait for teachers to get vaccinated, let alone for students to get vaccinated, before reopening.

Wait, how do you get this? Am I misreading the chart above this sentence? It looks to me like in the poll you quote, 55% thinks schools should wait, which seems in contradiction with your sentence.

The feeling of breaking an Overton window

Sometimes unconscious/visceral fears are kind of what I thought they were:

If I look down from a tall height, I experience: [vertigo, a slight increase in my heart rate, some similar change in my breathing, a slight increase in my tendency to "freeze" my muscles, a shift in my attention toward the height/downness, etc.].

I might've thought this response was a consciously chosen strategy for not falling. Except that it also occurs when I'm walking on a glass floor in a well-engineered building I trust. Still, I conceptualize it as something like "in-built fear of heights; designed to prevent falling but based partly on a bunch of visceral cues that persist even when my conscious mind knows I won't fall". My lead "rationalization" for the response ("I'm breathing more shallowly and refusing to walk to the edge of the cliff with you because I don't want to fall off the cliff") is at least partly post-hoc, and is causally downstream of a more visceral reaction... that is basically also evolved for not falling off cliffs.

And sometimes unconscious/visceral reactions are just not at all what I thought they were:

Some years ago, I was waiting impatiently for a friend to leave an event we'd both been at, so that I could "go home to do my important work with its urgent deadline." Then I took a better look at the feeling, and realized that I was actually cold and in need of a washroom. I found a washroom and some warmth, and felt suddenly at ease and untroubled about my task. I've related differently to such feelings ever since.

Similarly, some years ago I was hanging out with my cousins around a holiday, and I felt "bored" (as I saw it) "because" they weren't talking about anything interesting, and I couldn't share with them my thoughts about anything interesting, such as AI risk. And then we switched board games, and I started laughing more, and I realized that my previous "bored" has actually been made of "socially uncomfortable". And I got a bit better at identifying the "socially uncomfortable" visceral response, and calling it by that name instead of some other.

The interesting thing for me about the cashier incident above, was the rationalizations my brain produced around the "don't share my views" impulse in my cashier situation seemed pretty transparently unsound (in my situation, in contrast to e.g. Oliver's similar instance in-thread, or any number of previous times when I had more plausible ... rationalizations? reasons? for not breaking an Overton window). If I was concerned about the cashier's welfare: yes, she might find it uncomfortable to see our views, but "withholding practically relevant information from someone specifically asking about it" does not seem kind. If I was concerned about causing some sort of social drama that might do me harm: it didn't seem like there was much plausible harm. (The risk that she might object to selling me the groceries did not occur to me, I think correctly. The 24-hour Safeway just outside of Berkeley may have had different customers than the downtown Berkeley Trader Joe's that Oliver and Ben Pace mention. She seemed peaceable/stable/normal. I was with my husband, which is probably safer than alone. Etc.) There probably was still more social drama in telling the truth vs in evading, but its absolute amount seemed pretty small, to the point where the normal ratios at which I try to buy [human decency / communication / being helpful and honest] vs [avoiding harm to myself] seemed to pretty clearly favor talking.

So it was more a glass floor situation, than an actually being near a cliff situation. Useful for elucidating things. And I'm not yet sure what the analog of "avoid falling" is, that this reaction is actually triggered by cues of. Is there a pretty in-built visceral thing for "don't break Overton windows", that is quasi-independent of conscious knowledge that you're safe? Is its true name "don't break Overton windows", or something else? What's up with the way the impulse in me oscilated between selfish rationalizations ("she might harm me") and morality-related rationalizations ("it's wrong to upset people")?

The feeling of breaking an Overton window

Totally. I was not AFAICT worried at the time about limited supply buying, or not very worried; the Safeway we were getting things from did not seem out of much and I hadn't heard people complain about shortages/buying yet as far as I can recall.

“PR” is corrosive; “reputation” is not.

To be honest I am not sure what exactly is being advised.

I am basically advising that you treat the concept of PR, and the word “PR”, the way you would treat a skilled but incredibly sleazy used car salesman. You may sometimes wish to deal with him anyway, if you can’t practically locate any other way to buy a car. But you’ll want to be very very alert to what’s being slipped into “your” “beliefs”, while you do so.

Sort of like if you were using a concept from Scientology to navigate a personal psychological issue.

Do you think trying to be 'honorable' will suffice to avoid bad outcomes?

I think that attention to “honor”, “reputation”, “brand”, etc. will get us most but not all of what we might hope for from PR, and including some things that PR itself won’t give, such as some kinds of longer-term freedom, grounding, and ability to think.

I would advise using this concept first (just, very simply, substituting the word “reputational concerns” for “PR concerns” in conversations, and seeing where this substitution gets you).

I don’t think it’ll do everything PR would do. And I’m not saying you should never care about the residual (although I am saying that the sleazy car salesman may have tricked us into sometimes thinking the residual matters more than it does).

“PR” is corrosive; “reputation” is not.

It's much easier to resolve disagreements about what counts as good PR.

I mostly disagree. I mean, maybe this applies in comparison to “honor” (not sure), but I don’t think it applies in comparison to “reputation” in many of the relevant senses. A person or company could reasonably wish to maintain a reputation as a maker of solid products that don’t break, or as a reliable fact-checker, or some other such specific standard. And can reasonably resolve internal disagreements about what is and isn’t likely to maintain this reputation.

If it was actually easy to resolve disagreements about PR, I suspect we wouldn’t be so spooked by it, or so prone to deferring to outside “PR consultants”.

I… my thoughts aren’t coherent enough here to let me know how to write a short comment, so I’m gonna write a long one, while noting aloud that this is a bad sign about my models here.

But: it isn’t just a matter of deferring to polls. Partly because with “bad PR” or scandals, there’s a dynamicness to the mob. It isn’t about peoples’ fixed standards or comparisons, that you could get by consulting polls. (Or just by consulting a friend or two, the way you probably do when you are faced with normal ethical questions and you want help remembering what the usual standards are.) It’s some spooky other thing, involving dynamics that evolve, and experts that you pay to be a bit distanced somehow from your not-knowing and to be able to tell other people that of course you consulted an expert so that they won't shun you after the whole thing explodes.

It seems like it's easier for organizations to coordinate around PR

So, that does seem true, at least in the sense that lots of organizations and groups talk about “PR”; so there’s some tautological sense in which it’s gotta be easier for organizations to somehow end up talking about that. (Lately. It wasn't so in past centuries, FWIW. Possibly they just didn't know how.)

But I am a bit unclear on why.

One hypothesis that matches my own introspection, at least for small- to medium-sized organizations of the sort I’ve been involved in, is that we attend to PR because it’s somehow part of the received “everyone knows you should pay attention to PR” morality that was imparted to us, right next to “don’t drink and drive” and “get a college degree” and "remember to feel a sense of 'wow' if somebody mentions Harvard". And not because of inside-view/directly-perceivable advantages to attending to PR (vs reputation/brand/honor).

Of course, this just kicks the can down the road — why would this morality have been imparted to us? I’m honestly not sure. I don’t trust it. I do not personally notice myself or others making any worse decisions when we instead attend to "reputation".

“PR” is corrosive; “reputation” is not.

Thanks; I find this comment helpful and interesting, like part of a puzzle.

Is MIRI actually hiring and does Buck Shlegeris still work for you?

I'm not an official MIRI spokesperson here or something, but I'm a MIRI board member and I do a lot of work with MIRI (without being on payroll). My take, which someone else may turn out to correctly disagree:

  1. We are not hiring as much since the pandemic. We are however still hiring for some niche positions, especially ones that can easily be remote and/or that do not require the in-person internships for getting started that we often used.

  2. Separately, some but not all of our research programs are additionally not hiring as much lately for non-pandemic reasons, due to being in more of a "regroup and see what makes sense in terms of research directions, before possibly making hires and expanding again" place than they were last year. We should probably update our job ads but haven't been quite sure how and have been procrastinating therefore.

  3. Buck is mostly no longer working at MIRI but is doing some contracting, including about processing job applicants I think.

  4. Despite #1 and 2, I am aware of three hires made over the last few months, though all for niche positions assisting distinct people with distinct things. Also, various other bits of MIRI may turn out to be hiring at different times in ways that are hard for me and perhaps also them to predict. I do not have an official role in any of this, but if you'd like to talk 1-on-1 I'd be happy to, and it'd let me be aware of you if job openings later come up in parts of MIRI I'm connected to.

Still Not in Charge

If the social substrate people are in makes it easy to form binding contracts, people won't defect in prisoner dilemmas. Maybe I'm using the wrong words; I'm trying to agree with your point. I don't mean "coordination ability" to be a property just of the individuals; it's a property of them and their context.

Still Not in Charge

Yes; the test Zvi mentions seems like it actually tests "folks have utility functions and good coordination ability". (Like, good ability to form binding contracts, or make trades.)

Load More