Surviving Petrov Day

Do you think it makes more sense for you to punish the perpetrator after you're dead or after they're dead?

Replication is a decent strategy until secrets get involved, and this world runs on a lot of secrets that people will not back up. Even when it comes to publicly accessible things, there's a very thick and very ambiguous line between private data and public data. See, for example, the EU's right to be forgotten. This is a minor issue post-nuke, but it means gathering support for a backup effort will be difficult.

Access control is a decent strategy once you manage to set it up and figure out how to appropriately distribute trust. Trusting "your friends" is not a good strategy for exactly the reason evident today: even if they're benign, they can be compromised.

Punishing attackers just flat out doesn't work. That random person in China doesn't care if the US government says hacking is bad. Hackers don't care if selling credit card data is bad. Not even academic researchers care that reverse-engineering is illegal. You're not going to convince the world that your punishments are good, and everyone unconvinced will let it slide. All you'll do is alienate the people most capable of identifying flaws in your strategy. There are a lot of very intelligent people out there that care more about their freedom to explore and act than about net utility. They will build out the plans and infrastructure necessary for the real baddies to do their work. Please do not alienate them by telling them that their moral sensibilities are bad.

Some lessons from a decade in software security.

I like your backup strategies for LessWrong. The connection to nukes is tenuous. I think your Petrov Day strategy does more harm than good.

[-]Mati_Roy5y70

You seem angry that I wrote social penalty, and then seemingly punished me with -6 Internet points.

[-]sen5y*20

I did -2. It wasn't punishment, and definitely not for saying social penalty. I think social penalties are perfectly fine approaches for some problems, particularly ones where fuzzy coordination yields value greater than the complexity it entails.

I do feel frustration, but definitely not anger. The frustration is over the tenuous connection, which in my mind leads to a false sense of understanding.

I feel relatively new to LW so I'm still trying to figure out when I give a -1 and when I give a -2. I felt that the tenuous connection in combination with the net-negative advice warranted a -2.

EDIT: I undid my -2 in light of this comment thread.

[-]Mati_Roy5y20

Ok, thanks for clarifying.

FWIW, the connection was meant to be a playful one, not a serious one :)

I re-read your original comment. I'm understanding you're saying that I shouldn't alienate the person that did the social engineering because they didn't see this game the same way than I did (correct?) If you mentioned another way in which my post might have caused harm, I don't think I understood it. That said, you don't have to clarify; it's up to you; I'm fine either way :)

[-]sen5y10

Your understanding is correct. Your Petrov Day strategy is the only thing I believe causes harm in your post.

I'll see if I can figure out what exactly was frustrating about the post, but I can't make promises on my ability to introspect to that level or on my ability to remember the origins of my feelings last night.

These are the things I can say with high certainty:

I read this post more like a list of serious suggestions interspersed with playful bits. Minus the opener and the Information Flow section, the contents here are all legit.
If you put way more puns into the section contents, it would feel less frustrating.

This is a best-guess as to why the post feels frustrating:

It feels like you draw a sharp delineation between playful bits and serious suggestions. The opener is all playful. The section headers are all serious. Minus the Information Flow section, the section contents are all serious. The "Metaphor For" lines are all playful.
The sharp delineation makes it feel like the playful bits were tossed in to defend the serious suggestions against critical thinking.

This is a weak best-guess, which I could probably improve on if I spent an hour or so thinking about it:

I'd guess that puns would help because they would blur the line between serious suggestions and playful bits. This would force the reader to think more about what you're saying for validity. With that, it wouldn't feel like the post is trying to defend itself against critical thinking.

[-]Mati_Roy5y10

thanks for the clarifications!

[-]digital_carver5y20

I think 'Improving the Incentives' misses the point of Petrov day. The point of Stanislav Petrov is that he made his decision despite all his personal incentives being aligned the opposite way. It was a very hard thing to do, and creating incentives to not press the button cheapens the decision and takes away the difficulty of the decision - especially when in this circumstance the incentive to press the button is already a weak one (basically just "for the lulz" as far as I can tell), much weaker than the potential dangers to his career and his country that he had to deal with.

I see this Petrov day event as a chance to think about the difficulty of getting multiple people to do the "right thing" despite having little incentive to do so, about creating an environment that nurtures and nourishes such independent decision-making, and about the fragile balance of trust. Creating explicit incentives breaks away from this difficult problem and turns it into a different, often easier one.

[-]Mati_Roy5y10

That's a reasonable point.

On the other hand, if we develop a culture of retroactively punishing and rewarding people (ex.: Robin Hanson sent money to Stanislav Petrov), then maybe it will indeed create such incentives across the board (ie. in the real world as well).

[+][comment deleted]5y10

LESSWRONG
LW

LESSWRONG
LW

35

Surviving Petrov Day

35

35

Stopping the threat

Living through the winter

Recovering from a backup

Managing information flow

Improving the incentives