When Stanislav Petrov's missile alert system pinged, the world was not watching. Russia was not watching. Perhaps a number of superiors in the military were staying in the loop about Stanislav's outpost, waiting for updates. It wasn't theatre.

In contrast, LessWrong's historical Petrov Day celebrations have been pretty flashy affairs. Great big red buttons, intimidating countdown timers, and all that. That's probably not what the next "don't destroy the world" moment will look like.

It's also the case that some of the biggest moral dilemmas don't come clearly labeled as such, and don't have the options clearly marked as "cooperate" or "defect". (I think in Petrov's case, it was clear it was a big decision. Unclear to me how it easy it was for him to make and why.)

Matching the spirit of the above, this year's LessWrong commemoration was a little more one-on-one. It started with a poll. In previous year's, the LessWrong team has unilaterally decided the meaning of Petrov Day, often facing objections. So why not get a sense of what people actually think matters most?

We sent the following private message to anyone who'd been active on LessWrong in the previous 24 hours:

 

252 people responded to the survey at the time I started work on this post, and the results are pretty clear:

The Most Important Value of Petrov Day

Note: We did not actually spend much time thinking about the options in this poll, their framing, etc. Like under 10 minutes. Feel free to discuss in the comments.

VirtueNumPercent
Avoiding actions that noticeably increase the chance that civilization is destroyed14457%
Accurately reporting your epistemic state2711%
Quickly orienting to novel situations2510%
Resisting social pressure5622%
Total252100%

Results are not significantly different for users with 1000+ karma:

VirtueNumPercent
Avoiding actions that noticeably increase the chance that civilization is destroyed3549%
Accurately reporting your epistemic state811%
Quickly orienting to novel situations811%
Resisting social pressure5628%
Total107100%

 

Unilaterally pushing your own values over the collective?

I don't know whether what really was going on was genuinely idealistic as opposed to symmetrical fighting over resources, but a lot of the US<>Russia conflict seemed to be about values and beliefs about what was right. Capitalism, communism, etc.

This raises some good questions. What are the legitimate ways to promote your own values over other people? This is where the follow-up poll question took us.
 

Yes, we told everyone they were in the minority. It's a "game".

Users were divided on the most important virtue (we don't know their opinions on the other virtues listed re Petrov Day), but it seemed reasonable that next year we'd go with the majority (or at least plurality) as a focus.

However, part of the Petrov Day experience (imo) is individuals being options to unilaterally change how things go for everyone else. Such an option we did kindly provide.

After some discussion, the LessWrong team has decided to make the focus of next year's Petrov Day be the virtue that is selected as most important by the most people...

If you click the below link and are the first to do so of any minority group, we will make your selected virtue be the focus of next year's commemoration [instead].

The plain value choice, according me, is faced with a values difference (or belief difference?) to go along with the majority, or to decide that unilaterally you'll take the opportunity to promote what you think is correct.

I find myself thinking about Three Worlds Collide scenarios where you come across others with different values, and possibly there are power differentials. What do you do confronted by baby eaters people who prioritize communicating your epistemic status clearly above other things, and you have the power to defeat them?

It's interesting to think about.

Unilateralists

Here's what happened: 31 out of 181 users clicked the link to promote their own favored value over what the collective would have chosen.

Note that I sent out the second follow-up message in batches, and some people responded to the first message after the last batch, and they did not get this opportunity. 181 did. That's 17% of people willing to unilaterally promote their value.

I'm not sure of people's reasoning here. Oliver Habryka said he almost instinctively clicked the link because of how it was displayed. Many people click first, read later. Unfortunately, though I attempted to place the link in spoiler block, that didn't work.

Perhaps people reason that it's inevitable that someone clicks the link, so might as well be them. (But be the algorithm you want to see in the world??)

One user, Max H, did explain his reasoning in response to a shortform question I asked:

I got that exact message, and did click the link, about 1h after the timestamp of the message in my inbox.

Reasoning:

  • The initial poll doesn't actually mention that the results would be used to decide the topic of next year's Petrov Day. I think all the virtues are important, but if you want to have a day specifically focusing on one, it might make more sense to have the day focused on the least voted virtue (or just not the most-voted one), since it is more likely to be neglected.
  • I predict there was no outright majority (just a plurality) in the original poll. So most likely, the only thing the first clicker is deciding is going with the will of something like a 20% minority group instead of a 30% minority group.
  • I predict that if you ran a ranked-choice poll that was explicitly on which virtue to make the next Petrov Day about, the plurality winner of the original poll would not win.

All of these reasons are independent of my actual initial choice, and seem like the kind of thing that an actual majority of the initial poll respondents might agree with me about. And it actually seems preferable (or at least not harmful) if one of the other minorities gets selected instead, i.e. my actual preference ordering for what next year's Petrov Day should be about is (my own choice) > (one of the other two minority options) > (whatever the original plurality selection was).

If lots of other people have a similar preference ordering, then it's better for most people if anyone clicks the link, and if you happen to be the first clicker, you get a bonus of having your own personal favorite choice selected.

(Another prediction, less confident than my first two: I was not the first clicker, but the first clicker was also someone who initially chose the "Avoiding actions..." virtue in the first poll.)

He thought about it! I won't dive into discussing this here, but curious to hear from other link-receivers why they did or didn't click.

Which virtue-promoter group is the most unilateralist-y?

Before you hover/click into the spoiler text, please take a moment to register your predictions about which Virtue group clicked the "unilaterally make your virtue the focus" button the most.

 

VirtueTotal in GroupNum Link ClickersPercent of Group
Avoiding actions that noticeably increase the chance that civilization is destroyed1041716%
Accurately reporting your epistemic state19421%
Quickly orienting to novel situations17212%
Resisting social pressure40820%
Total 31 

 Well, that really does make a lot of sense, frankly. Watch out for them nonconformist types, they don't care what you want!

EDIT: Oops, no, data error. I mismatched the values and it is not the case that "Resist social pressure" group "defects" at a higher rate.

So which virtue actually wins?

The very first click was from someone in group A, after 1 minute and 31 seconds had elapsed since they received the message.

But I think it's unfair to base this on literally the first click, since the message contain the link was sent out in a very staggered way, in a few rounds.

In the first round, Group A got the message several minutes earlier than B, then C, etc. Then there was a subsequent round. Group A is the largest so might have had a clicker.

We can look here and see for each group, how long did the people who clicked the unilateralist link take to do so after they received the message.

Dun dun dun...

And well, fittingly perhaps, the person who clicked first in absolute terms and the person who clicked first relative to receiving are the same person! And also that person is from the majority group Avoiding actions that noticeably increase the chance civilization is destroyed.

And since they are from the majority group and they clicked the link, they and thier group are disqualified! Meaning we go with the second largest group, Resisting social pressure! Congrat Group D! You're the most defect-y, but you win.

No, I joke. There's no disqualification of your entire group. If I have say in next year's LessWrong celebrations, I think we should honor our word[1] and go with the majority/person who clicked the link first, which is Avoiding actions that noticeably increase the chance civilization is destroyed!!!! Woooo. Good work.

That's the most important message of Petrov Day. An absolute majority of 57% of respondents confirm it.

EDIT: Vanessay Kosoy points out in the comments that what I actually wrote was:

If you click the below link and are the first to do so of any minority group, we will make your selected virtue be the focus of next year's commemoration.

And since many people clicked, the winner should be the first click from a minority group. This would be someone who selected Accurately reporting your epistemic state.

I find this reasoning compelling, but will allow myself to think/hear other arguments.


In all seriousness, I am pretty interested in question of how to behave when you're sharing resources with people with differing values, including resources that determine which values get promoted more.

Majority vote is a basic standard of fairness, but maybe if you can "cure" the babyeaters, you do so by force when given the chance. Or there are lots of other things that play into it, as Max H's reasoning is above. I'm quite curious, please share your experience and thoughts from the other end.

 

  1. ^

    I twinge slightly that for the sake of the "game" we sent the majority group a message falsely saying they were in the minority. This was mostly that it was faster to not special-case it.

New to LessWrong?

New Comment
73 comments, sorted by Click to highlight new comments since: Today at 2:19 PM
Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings

This was a really great Petrov day thing-to-do. I clicked on the link basically instinctively, after having a panic about being under time pressure, and about having my preferred outcome not taken, and therefore not being able to think so well about what my actions would actually do. Immediately after clicking, I felt very good, and like I had made the right decision, and then I felt a sinking sensation in my gut as I realized that if I had not taken any action, and nobody else had taken any action, then in a year I'd get to see what LessWrong as a collective decided, and I realized that actually I really wanted to see the result of that Petrov day, even if it meant Petrov day wouldn't be about preventing the end of the world.

For the rest of that day, and into today, I could not be in the world where LessWrong both collectively voted on what they wanted, and collectively refused to overrule the majority. It felt like I had ruined Petrov day. Even if others had still clicked the link in my stead, I still didn't want to have been among the people who ruined Petrov day, and it would feel even worse if I was the first, which I place nontrivial probability on (according to Firefox's his... (read more)

5Ruby7mo
Thanks for sharing all of that in such detail, <3 You make me feel quite glad we did this celebration. Would you like to know which number click yours was?
2Garrett Baker7mo
I do want to know
4Ruby7mo
You were the first, as you guessed.

I picked "resisting social pressure" and then when I got the second message, I thought "Aha, I was asked if I value resisting social pressure, and now I'm offered the chance of applying social pressure to make things go my way, to see if I will defect against the very virtue I claimed to be in favor of! I'm guessing that there's a different message tailored for each of the virtues, where everyone is offered some action that is actually the opposite of the virtue they claimed to endorse, to see how many people are consistent. Clever! Can't wait to see what the opposite choice for the other virtues is."

Now I'm slightly disappointed that this wasn't the case.

6quetzal_rainbow7mo
I would be worried about opposite action for "avoiding to increase probability of doom"...
2Pee Doom7mo
Similar for me. I was very suspicious at first that the first message was a Scam and if I clicked I would blow up the website or something tricksy. Then with the second message I thought it might be customized to test my chosen virtue, "resisting social pressure", so I didn't click it.
1Dweomite7mo
I don't think it's inherently contradictory to apply social pressure while thinking that resisting social pressure is a virtue. One possible reason for thinking that resisting social pressure is a virtue is if you think social pressure is generically bad, in which case applying it does indeed seem hypocritical.  But it seems like there could be other reasons for considering that virtuous. Perhaps you think that resisting social pressure shows personal strength and conviction, and that applying social pressure helps people to train their strength and conviction. Or perhaps you think the world is made of a mixture of naturally-good people who can resist social pressure and naturally-bad people who can't, and given that the bad people exist it's pragmatically important to ensure that net social pressure points in a prosocial direction. (In case it changes how you interpret my reasoning:  I chose not to vote in the virtue poll.)

I saw the poll, found it really yucky, and thus didn't answer it. I like LW for mostly staying on low simulacrum levels, and this poll felt like anything but. In more detail:

"Today is Petrov Day! A day celebrating that the world didn’t end, and the virtues that helped it not end. There’s been a lot of discussion." [...]

Virtue A – Avoiding actions that noticeably increase the chance that civilization is destroyed [...]

Virtue D – Resisting social pressure

  1. So the poll begins with an explanation of Petrov Day, and then leads with that as the first answer. This felt like one of those television thingies where they aren't allowed to just enter you into a giveaway, and instead make you answer a question like "The president of the US is X. Who is the president of the US?", so that it's considered a game of skill, rather than pure chance. This kind of yucky Dark Arts style of leading questions is common, but I expect much more from LW.
    1. Another leading / biasing aspect: Virtue A is by far the longest answer, and on my screen is the only one that covers two lines rather than one.
  2. Anyway, having decided that I don't like the poll, there's no option for me to pick. Since the poll is so transparent
... (read more)
3Ruby7mo
I'm sad you didn't like it. It indeed was not a carefully planned rigorous survey of Petrov Day attitudes. In my thinking, it was more the start to a ~game/exercise than trying to maximally model people's attitudes. I wanted to assign people to "teams" (I'd considered random assignment), but this felt this was a little more meaningful, and there's non-zero bits even in an imperfect survey. There was no intention to be leading in the responses, nor to corral for any particular response. I actually hoped that the slap-dash nature would make people suspicious (plus inadvertent bugs/typos) and get them more into Petrov Day mood. From other comments, it sounds like this did happen somewhat. I think if a failure happened here, it's that you and others saw the poll as primarily an attempt to accurate survey LessWrong member's beliefs about Petrov (pretty reasonable belief), but for me it was the start to something else, and goal wasn't "rigorous survey", for which a mu option would have made sense. I'm uncertain how much we should ever be a little sneaky/misleading for the sake of games/experiments/etc. I'm a pro norm that on April Fool's and Petrov Day and similar, people might hoodwink you a little. At least I might, I will as say a matter of metahonesty.
2MondSemmel7mo
Sure, I get that. But the result also matters. I predict that, if you presented the parts I quoted from the survey message to a random sample of the university-educated population, and asked them whether they thought the poll was biased, >50% would say yes. Doubly so if you also showed them the lopsided survey results. Anyway, if the first message had been presented as a pick-your-team exercise, I indeed would've been somewhat less frustrated with it. Though even then, I don't think the team category "resist social pressure" or "be contrarian" ever works; it's an inherent contradiction. And as such, and in keeping with the Petrov Day theme, I maintain that it's important to offer a true "mu" option, or a "do not participate". And insofar as one is inevitably tempted to interpret the results, while you do mention "there's non-zero bits even in an imperfect survey", your post does some interpretation but imo without sufficiently acknowledging the imperfection, which I don't think is particularly conducive to painting an epistemically accurate picture. For example, without the "mu" option, we don't even know which fraction of LWers who saw that message actually picked an option!
2Ruby7mo
That seems quite plausible to me. My response was that we weren't trying especially hard to avoid bias because we weren't trying to get a super clear result. Can you elaborate on this? 
3MondSemmel7mo
Petrov Day has a variety of themes. A few comments here endorse the unilateralist interpretation, but insofar as another thematic connection is conscientious objection or resisting social pressure, the following argument applies: You offered the option "resist social pressure". As I argued, this option doesn't really work, in that picking it doesn't really mean what it sounds like. It's like giving someone the options "think inside the box" or "think outside the box". Then choosing the latter is not a sign of creativity, just like choosing the option "resist social pressure" is not sufficiently (honoring the virtue of) resisting social pressure, because by picking it you're ultimately still conforming to the frame you've been given. I don't know if this point resonates, but I feel pretty strongly about it. (Relatedly, it's in some sense harder to resist the social pressure of your own peer group than that of the broader society.) Anyway, due to the above, if you want an option to actually symbolize <resist social pressure>, then options like "mu" or "do not participate" imo work much better than the explicit description "resist social pressure", because these alternatives let people answer without needing to buy into the provided frame.
2Ruby7mo
Sorry, I get the point that the option provided doesn't let you mu/reject the frame. It's not clear to me that this is a core framing of Petrov's actions/virtue was conscientious objection or so on. Beyond that, the survey wasn't aiming to allow people to symbolically act out their responses or to reject the frame in an unambiguous way. Insisting that you get to register that you saw it but didn't like it feels like insisting that you get to participate, but in your own way, rather than simply not engaging if you don't like it. I also feel like if there was an option to conscientious object and you took that, that'd still be within the frame I created for you to do so? But open to being corrected here.
2MondSemmel7mo
Well, personally speaking, I've already spent more time on my comments here than were warranted by my initial annoyance. So if I consider just myself, then choosing not to vote, followed by registering that non-vote in a comment, was of course a sufficient (albeit time-consuming) way to object. However, looking at this comment thread by Said shows that besides me, at least three other people (EDIT: here's one more) had problems with the framing and hence didn't vote. But because there was no "mu" option, we'll never know whether the proportion of those people was 0.1%, 1%, 10%, 50%, 90%, or whatever. That I continue to find unfortunate, and a "mu" option would've remedied this problem.
2Ruby7mo
If I were to do it again, I might include such an option, though I'm still not terribly sad I didn't. If we really wanted that info, I could  sample and survey people who received the message, looked at it (we have data to know this) and ask them why they didn't vote. My guess is between 1-10% of people who didn't vote because of frame commented about it,, so that's 40 to 400.  372 people have responded by now out of 2500, so 15%. Let's guess that 50% of people saw it by now, so ~1250 (though could get better data on it).  A third responded if so, which seems great for a poll. Of those the 800 who saw but didn't, I could see 100-400 doing so because the frame didn't really seem right (lining up with the above estimate). Which seems fine. I bet if I'd spent 10x developing the poll, I wouldn't get that number down much, and knowing it with more precision doesn't really help. It's LW, people are very picky and precise (a virtue...but also makes having some nice things hard).

I didn’t participate in the poll because it didn’t make much sense, the options didn’t seem to cover the possibilities or really contain what I’d pick if asked in free-response form, and the whole thing seemed rather slapdash. Clicking any of the options wouldn’t’ve reliably communicated anything about my beliefs.

(Also, there was a spelling error in the poll option URLs, which said “petroy” instead of “petrov”. This further undermined any confidence I may have had in the poll meaning anything.)

4Ruby7mo
I'm curious to know what your free-form response would be.
8Said Achmiz7mo
Something like: “What does ‘would preserve that [virtue] over all others’ mean, exactly? What’s the scenario here, what’s forcing me to make this choice and what exactly do I think will be the consequences of making the selection? Who am I ‘reporting’ my epistemic state to? Why is there no option that talks about seeking the truth and understanding reality? Can I pick ‘the virtue of not taking silly philosophy questions at face value’?”
3David Hornbein7mo
At the risk of butting in, I also didn't participate because none of the options reflected my own views on the important virtue of Petrov Day, which is more like "Do not start a fucking nuclear war". You can try to distill that down to nicely categorized principles and virtues, and some of those might well be good things on their own, but at this level of abstraction it doesn't capture what's special about Petrov Day to me. Trying to file down the Petrov Day narrative into supporting some other hobbyhorse, even if it's a wonderful hobbyhorse which I otherwise support like resisting social pressure, is a disservice to what Stanislav Petrov actually did. The world is richer and more complex than that. I personally preferred the past Petrov Day events, with the button games and standoffs between different groups and all that. They didn't perfectly reflect the exact dilemma Petrov faced, sure, but what could. They were messy and idiosyncratic and turned on weird specific details. That feels like a much closer reflection of what makes Petrov's story compelling, to me. Maybe the later stages of this year's event would've felt more like that, if I'd seen them at the time, but reading the description I suspect probably not. I like that you guys are trying a bunch of different stuff and it's fine if this one thing didn't land for me.
3vitaliya7mo
That's why rather than clicking on any of the actual options I edited the URL to submit for choice=E, but as per the follow-up message it seems to have defaulted to the "resisting social pressure" option. Which... I guess I was doing by trying to choose an option that wasn't present.
2Ruby7mo
Huh, I'm surprised that happened. I wouldn't have thought you'd get a message given that.
2Adele Lopez7mo
I also incorrectly got the follow-up "resisting social pressure" option. (My original choice was A)
[-]Firinn7mo120

I think answering "how should you behave when you're sharing resources with people with different values?" is one of the projects of contractarian ethics, which is why I'm a fan.

A known problem in contractarian ethics is how people with more altruistic preferences can get screwed over by egalitarian procedures that give everyone's preferences equal weight (like simple majority votes). For example, imagine the options in the poll were "A: give one ice cream to everyone" and "B: give two ice creams, only to the people whose names begin with consonants". If Selfish Sally is in the minority, she'll probably defect because she wants ice cream. When Altruistic Ally is in the minority, she reasons that more total utility is created by option B - since consonant names are in the majority, and they get twice as much ice cream - so she won't defect and she'll miss out on ice cream. Maybe she's even totally fine with this outcome, because she has tuistic preferences (she prefers other people to be happy, not as a way of negotiating with them, simply as an end-in-itself) satisfied by giving Sally ice cream. But maybe this implies that, iterated over many such games, nice altruistic kind people... (read more)

[-]Raemon7mo110

I'm the one who picked the particular virtues we displayed, and it seemed good to explain a bit more about where I was coming from. We've had a lot of interesting discussion each year about the True Meaning of Petrov Day. 

Eliezer's original post on Petrov Day begins:

Today is September 26th, Petrov Day, celebrated to honor the deed of Stanislav Yevgrafovich Petrov on September 26th, 1983.  

Wherever you are, whatever you're doing, take a minute to not destroy the world.

The theme of "Reflect on the fact that the world could have been destroyed, and be the sort of people who don't destroy the world" was a central element most Petrov Day celebrations for years going forward. In 2019, when the LW team first built The Button experiment for LessWrong (where some users could press a button that took down the LessWrong frontpage for a day), we leaned heavily into this framing.

But throughout the years, some people have objected to this being the primary (or at least only) framing of the event. We've had a lot of interesting discussion each year about the True Meaning of Petrov Day. 

Some people argued that it's actually kind of confused to celebrate Petrov for the sake of "... (read more)

3Raemon7mo
Also, after shipping the initial code and initial wave of notifications, it took us a while to realize I had mispelled "Petrov" as "Petroy". We thought about trying to abort the notification volley, but decided "well, figuring out whether the system is malfunctioning is just a kinda reasonable part of the game."

First, I clicked the link in the second poll[1]. My thought process looked as follows:

  • I quickly skimmed the content of the message
  • My split-second-judgement registered that there is a RACE
  • Moreover, the race is on very small time scales: every second of indecision might cost me victory!
  • Moreover, split-second-judgment estimates that winning the race is good-in-expectation (where "expectation" should be thought of as including the "logical uncertainty" resulting from having to rely on split-second-judgement).
  • Therefore, click NOW before it's too late!

Worse, even after clicking and reading the text again, I misunderstood its content. Somehow, I thought that this year's celebration will be determined by the plurality, whereas next year's will be determined by the fastest minority. This system is strange, but is not obviously defect-y, i.e. not obviously inferior to e.g. using plurality twice in a row, from behind the veil-of-ignorance.

Only after reading the OP and starting composing this comment in my mind, did I understand the actual meaning of the text in the second poll: that only the next year's celebration is decided upon, and only according to a minority (if anyone in a minority cli... (read more)

6Ruby7mo
You make a very good point! I think I should update here. I too have been acting in haste. While in past years we spent quite significant number of person-days on Petrov Day, this year we've been focused elsewhere so this post was quickly written too. Fortunately, it gets feedback. Thanks, and I'll update the OP to at least say I'll need to review the decision here.
[-]JBlack7mo124

How can you honour your word at all if the premise of the link was false for more than half the respondents? There is no action that is consistent with your words.

4Ninety-Three7mo
It seems quite easy to me. Imagine me stating "The sky is purple, if you come to the party I'll introduce you to Alice." If you come to the party then me performing the promised introduction honours a commitment I made, even though I also lied to you.
3JBlack7mo
A closer analogy is "You are an interesting person, and I will introduce the first interesting person who comes to the party to Alice". You come to the party, you're told that you're the first there, but you're not introduced to Alice because you're not an interesting person after all. Instead they introduce the first interesting person to Alice (who for some reason only has time to meet one person). Ah never mind, I now see what you meant. Yes in general you can narrowly honour your commitment by carrying out the action, but I mean more by "honouring your word" than just that. As I see it, someone who deliberately lies has not honoured their word, regardless of any subsequent actions that they might perform. They've made two statements, one vouching that something is true, and one vouching that something will be true. Ensuring that the latter will be true does nothing to restore their loss of honour from the deliberate falsity of the former. In this case they can't even honour the latter part, since they made a mutually exclusive promise to two different people.
1Ben Smith7mo
Seems to me that in this case, the two are connected. If I falsely believed my group was in the minority, I might refrain from clicking the button out of a sense of fairness or deference to the majority group.  Consequently, the lie not only influenced people who clicked the button, it perhaps also influenced people who did not. So due to the false premise on which the second survey was based, it should be disregarded altogether. To not disregard would be to have obtained by fraud or trickery a result that is disadvantageous to all the majority group members who chose not to click, falsely believing their view was a minority. I think, morally speaking, avoiding disadvantaging participants through fraud is more important than honoring your word to their competitors. The key difference between this and the example is that there's a connection between the lie and the promise.
4Raemon7mo
I really like this takeaway, and generally like how "rationality test self-assessment" process here.
[-]Ann7mo9-1

Reasoning through the consequences of removing all virtues but the chosen one, an inability to avoid actions that lead to noticeable increase that civilization was destroyed tended to outweigh the others in expected negatives, so I picked that. It's also traditional, and is the one that makes any sense for the day to be about, though that wasn't what I was asked. I also thought through how the poll link might be a red button in some way but didn't see it.

I was quite puzzled by the second message that my choice to maintain the traditional and expected value... (read more)

I voted for correctly reporting your epistemic state. I claim that this is the actual virtue Petrov displayed, and that his primary virtue being "don't take actions which destroy the world" because he decided to buck the chain of command is a mistaken belief. From the Wikipedia article:

Petrov later indicated that the influences on his decision included that he had been told a US strike would be all-out, so five missiles seemed an illogical start;[3] that the launch detection system was new and, in his view, not yet wholly trustworthy; that the message pass

... (read more)
2Richard_Kennaway7mo
The poll did not ask what virtue Petrov most displayed. It asked what virtue you think is most important.
2ryan_b7mo
Yes, and the virtue that is most important is the one that allowed Petrov to not doom the world. By contrast, the two most popular choices were about refusing to doom the world, and resisting social pressure, neither of which were features of the event. If there was a poll in connection to Arkhipov, my answer might change.

Yes, we told everyone they were in the minority. It's a "game".

 

I think this is bad. I mean, it's not that big a deal, but I generally speaking expect messages I receive from The LessWrong Team to not tell falsehoods.

4Ruby7mo
My metahonesty is I might hoodwink you a little on April Fools, Petrov Day, and similar.
1Quinn7mo
Indeed - but maybe I was overly paranoid and I should've just clicked the buttons on the day.
3Raemon7mo
My current moral code says "it's okay to do things like this, as part of games, psychology study-ish-things, jokes, etc", if you tell people the truth shortly afterwards (which we did).
4Ninety-Three7mo
Separate from the moral issue, this is the kind of trick you can only pull once. I assume that almost everyone who received the "your selected response is currently in the minority" message believed it, that will not be the case next year.
2Raemon7mo
Yup, seems correct.

Hmm.

I don't think Avoiding actions that noticeably increase the chance civilization is destroyed is necessarily the most practically-relevant virtue, for most people, but it does seem to me like it's the point of Petrov day in particular. If we're recognizing Petrov as a person, I'd say that was Petrov's key virtue.

Or maybe I'd say something like "not doing very harmful acts despite incentives to do so"—I think "resisting social pressure" isn't quite on the mark, but I think it is important to Petrov day that there were strong incentives against what Petrov did.

I think other virtues are worth celebrating, but I think I'd want to recognize them on different holidays.

I clicked the link in the second email quite quickly--i assumed it was a game/joke, and wanted to see what would happen. If I'd actually thought I was overriding people's preferences, I... probably would have still clicked because I don't think I place enormous value on people's preferences for holiday reasons, and I would have enjoyed being the person who determined it.

There are definitely many circumstances where I wouldn't unilaterally override a majority. I should probably try to figure out what the principles behind those are.

1CBiddulph7mo
Came here to say this - I also clicked the link because I wanted to see what would happen. I wouldn't have done it if I hadn't already assumed it was a social experiment.

In the first poll I voted "Accurately reporting your epistemic state" because I feel like it is systemically important and sort of a foundation on which other things like not destroying the world can be built (e.g. avoiding actions that noticeably may lead to the destruction of the world is more effective if you are better at sharing information relevant to which actions do or do not destroy the world). However if I had known that it was a poll about the next Petrov day celebration, I would have voted "Avoiding actions that noticeably increase the chance that civilization is destroyed" because I feel like that's the point of Petrov day.

clicked first relative to receiving are the same person! And also that person is from the majority group

A majority member being the initial clicker also isn't terribly surprising because a group being larger means one-or-more of any given sort of person -- in this case, a quick-responder-type -- is likelier to crop up among them.

I would be curious to see what the poll results for Question 1 look like, say, a week from now. I only saw the message in my inbox after Petrov day was over, and still responded.

[-]Max H7mo3-1

Ah, for the purposes of responding to the PMs (and writing my explanatory comment) I accepted the truth of the setup mostly without question, but it makes more sense in hindsight that the "Avoiding actions..." virtue was actually a strong majority vote.

About what to do about next Petrov Day, I mostly agree with Vanessa's reasoning, but I don't think it matters too much either way and don't really think it's about "honoring your word".

Once the truth of the setup is violated, it's mostly just about whether you want to do the thing that predictably would have... (read more)

I selected "Quickly orienting to novel situations" (QOTNoS) because it's strictly superior to the alternatives:

  • If you have the QOTNoS virtue, you can deal with the novel situation of AI destroying civilization
  • The necessity of "accurately reporting your epistemic state" is a novel situation for most people. QOTNoS helps again.
  • "Resisting social pressure" is a common situation. But if the survival of civilization depends on it (as the poll implies), this is a novel situation. Thus, QOTNoS will help with it.

In essence, QOTNoS (as in being able to make the right decisions in novel situations) is a synonym for general intelligence, and thus is the strongest power. 

4Raemon7mo
Yeah I also personal picked 'quickly orient' for these reasons. 

Okay, I'm one of the unilateralists and I totally haven't thought about poll as "value" choice. I've perceived poll as if it was: "Which ability is the most important to prevent destruction of the world? 1. INT 2. WIS 3. CHA 4. LUK", and question "which virtue has the most impact on chance of world not being destroyed" is a factual question, not value question. If I thought about poll more seriously, I would consider "I'm in minority on factual question on site with smarter than me people, maybe I'm wrong", but I was pretty confident and mostly took it as fun activity, not Very Serious thing.

Also, it can be the matter of language differences: English is not my native, maybe we have different connotations around word "virtue".

To me, the initial poll options make no sense without each other. For example, "avoid danger" and "communicate beliefs" don't make sense without each other [in context of society].

If people can't communicate (report epistemic state), "avoid danger" may not help or be based on 100% biased opinions on what's dangerous.

  • If some people solve Alignment, but don't communicate, humanity may perish due to not building a safe AGI.
  • If nobody solves Alignment, but nobody communicates about Alignment, humanity may perish because careless actors build an unsafe AGI witho
... (read more)

I never registered that I was being asked to make a decision on behalf of the whole site. I might have considered that interpretation of your second question for a fraction of a second, but then dismissed it as extremely unlikely to be your actual intent.

I spent probably less than 3 seconds on the second question (message). To get me to spend more time than that, your first 25 words or so would have needed to include some sign that something significant was at stake in my response. "I'm prepared to send you 5 dollars," would have done it. So would signs that you sent the message only to me (or only to me and 4 other people) rather than to everyone on the site. (I was able to deduce the latter somehow.)

[-]Ruby7mo20

I didn't list in the main post or say until because I fear that I'm saying it defensively in response to criticism, but to model the design for this year requires knowing that we spent vastly less time on it, deciding to something at the last minute. (We'd been very busy with a massive conference in days before Petrov Day.

At 11am (US West Coast time) we started thinking there was something we could maybe do, and at 12pm we got started. I felt we needed to rush if we were to include European folks at all, so really was looking for something we could get don... (read more)

Registering my predictions for which groups clicked the second link most:

Percentagewise, I don't Groups A and C clicked on it that much (though I'd be surprised if the number from each group isn't non-zero), since they picked a choice that indicates that they care about making high-quality decisions and cooperating with the rest of the world. A higher proportion of C probably clicked than A, since a person might decide it's worth it even if they take their time to think it through (I'd disagree, but the commentor you quote fits into that category).

I'd then... (read more)

This is not responding to the interesting part of the post, but I did not vote in the poll because I felt like virtue A was a mangled form of the thing I care about for Petrov Day, and non-voting was the closest I could come to fouling my ballot in protest.

To me Petrov Day is about having a button labeled "destroy world" and choosing not to press it. Virtue A as described in the poll is about having a button labeled "maybe destroy world, I dunno, are you feeling lucky?" and choosing not to press it. This is a different definition which seems to have been e... (read more)

5Raemon7mo
It feels fairly important to me that in real life, Petrov had a "maybe destroy world, are you feeling lucky?" button. (It sounds like we disagree on this?) Like, relaying information up the chain of command a) doesn't automatically mean that they launch a full scale counterattack, b) that doesn't mean the US automatically launches a full scale counterattack, c) my current belief is that full scale nuclear war probably cripples the northern hemisphere but doesn't literally kill all humans (which is what I think most people mean by 'destroy the world')
4Ninety-Three7mo
Petrov was not the last link in the chain of launch authorization which means that his action wasn't guaranteed to destroy the world since someone further down the chain might have cast the same veto he did. So technically yes, Petrov was pushing a button labeled "destroy the world if my superior also thinks these missiles are real, otherwise do nothing". For this reason I think Vasily Arkhipov day would be better, but too late to change now.  But I think that if the missiles had been launched, that destroys the world (which I use as shorthand for destroying less than literally all humans, as in "The game Fallout is set in the year 2161 after the world was destroyed by nuclear war), and there is a very important difference between Petrov evaluating the uncertainty of "this is the button designed to destroy the world, which technically might get vetoed by my boss" and e.g. a nuclear scientist who has model uncertainty about the physics of igniting the planet's atmosphere (which yes, actual scientists ruled out years before the first test, but the hypothetical scientist works great for illustrative purposes). In Petrov's case, nothing good can ever come of hitting the button except perhaps selfishly, in that he might avoid personal punishment for failing in his button-hitting duties.
3Raemon7mo
(I edited in more reply you may want to respond to. I think the button wasn't actually designed to "destroy world", it was designed to launch a counterattack. Petrov did seem to think it would based on some other quotes of his, but, like, AFAICT he was wrong. I think this is also true for Arkipov)
2Ninety-Three7mo
Granting for the sake of argument that launching the missiles might not have triggered full-scale nuclear war, or that one might wish to define "destroy the world" in a way that is not met by most full-scale nuclear wars, I am still dissatisfied with virtue A because I think an important part of Petrov's situation was that whatever you think the button did, it's really hard to find an upside to pushing it, whereas virtue A has been broadened to cover situations that are merely net bad, but where one could imagine arguments for pushing the button. My initial post framing it in terms of certainty may have been poorly phrased.
4Ruby7mo
There is an upside to being the kind of person who will press the button in retaliation. You hope never to, but the fact that you credibly would allows for MAD game theory to apply. (FDT, etc. etc.)

Was everyone supposed to receive the second message? I only got the poll and not the second message with the unilateralist link.

2Morpheus7mo
Ah I see:
[-]Ansel7mo2-1

I modestly propose that eating babies is more likely to have good outcomes, including with regard to the likelihood of apocalypse, compared to the literal stated goal of avoiding the apocalypse.

1Stephen Bennett7mo
This seems like a fairly hot take on a throwaway tangent in the parent post, so I'm very confused why you posted it. My current top contender is that it was a joke I didn't get, but I'm very low confidence in that.
2Ansel7mo
The parent post amusingly equated "accurately communicating your epistemic status", which is the value I selected in the poll, with eating babies. So I adopted that euphemism (dysphemism?) in my tongue-in-cheek response. Also, this: https://en.wikipedia.org/wiki/A_Modest_Proposal

A Review of Petrov Day 2023, according to the four virtues. First a check on the Manifold predictions for the day:

Avoiding actions that noticeably increase the chance that civilization is destroyed

LessWrong avoided creating a big red button that represents destroying civilization. This is symbolic of Virtue A actions like "don't create nuclear weapons" and "don't create a superintelligenc... (read more)

This was fun thank you!

There must be a hiccup in the data because you show < 30 total group b pickers in the first chart, but say there are 40 group b pickers in the "unilaterally make your virtue the focus" chart.

2Ruby7mo
Oh, good catch. I had the rows on the denominator sorted wrong so that table was 75% wrong. Fixed now...

Rule 1: Don't destroy the world

6Richard_Kennaway7mo
Rule 0: Cultivate those virtues that will stand you in good stead when you are faced with an opportunity to not destroy the world. Virtue A is the most important outcome, but B,C, and D are part of what it takes to achieve A.