A quick retrospective on the four Manifold markets on the day, focusing on the possibility of good or bad incentives flowing out of the markets. Several people expressed concerns that the Manifold markets would cause LessWrong's home page to go down, or go down earlier, due to incentives. People also placed bets and limit orders to try to mute those bad incentives. Some examples:
"This is a classical example where having a prediction market creates really bad incentives." "I think the right thing to do here is buy yes so that it's more beneficial for others to buy and hold no." "I'm not wild about this market existing, but given that it does exist I'm strongly in favour of making it profitable for others to not press the button."
I decided to look into this. I want to say up front that I think all the incentives were very small relative to perceived stakes, and I have no suspicions of anyone after writing this comment. I am also not going to give any bettor names.
By the end of the YES/NO market, several users had bet large sums on YES and were consequently incentivized to blow up the home page. One user had m26,002 YES shares (USD $260). Another user had 7328 NO shares (USD $73) and was incentivized not to blow up the site. But this doesn't show the whole picture, because there were also limit orders. As was explained to me:
The choice of whether to leave (public) limit orders on one side or the other is the way you incentivize action.
At various points during the day there were large (m1000, etc) YES limit orders in play. This is completely useless as an incentive for an individual LessWrong user. Sure, if I was the only one who might press the button, I could bet through those limit orders, not press the button, and collect my bounty. But many people could press the button, and this would have lost me mana. They did count as an incentive for LessWrong coders! They could quietly introduce a bug preventing the button from being pressed, bet NO, and collect their winnings. Nobody took up that incentive.
I didn't see any large NO limit orders. A NO limit order on a market like this is inherently risky, because if the button is pressed the offer will definitely be taken, but if the button is not pressed it probably won't. If there had been then that could have added to the incentive some bettors had to blow up the home page, but as far as I can tell there weren't.
The positions in the WHEN market were much smaller, with the largest positions coming in at 552 shares in LATE/NEVER and 325 shares in EARLY. The current design of range markets on Manifold is fun to play with, but is not very "swingy". In particular, there is no option for outsized winnings if you can predict exactly when something will happen. If a user was planning to blow up the home page in the last few hours they could have bet a combination of LATE/NEVER in this market and YES in the binary market to maximize their gains, but I don't see any evidence of this. Because there are fewer high karma users, large EARLY bets in this market could slightly limit their incentive to bet EARLY and blow up the home page early. Unlike the YES/NO market, those bets didn't happen.
If someone was willing to anonymously blow up the home page, then a pattern of placing huge bets on that happening might look suspicious and damage their anonymity. So generally I would expect these incentives to only be effective for someone who was willing to blow up the home page and take credit for it. You can bet on that happening in Will anyone try to take credit for nuking LessWrong?. Or, place a limit NO order to incentivize someone to bet YES and then take the credit (and the mana).
When I created my market - Will LessWrong change their Petrov Day 2022 plan and reduce the chance of the button being pressed? - it was swiftly bet down to a low probability, creating a small incentive for the LessWrong admins to bet YES and then change their plans. Nobody took up that incentive, and the market didn't attract large limit orders.
Finally mkualquiera made a market - Will my friend agree to defect on Petrov Day?. This market is evidence that two people were thinking about being incentivized to blow up the home page by betting markets. But it seems like this was more about a fun game than making mana. The largest position was 301 shares on YES, and the market resolved NO, so whatever incentive those shares provided, they weren't enough to make it happen.
In the end I think there is a low (10%) chance that anyone's behavior was significantly shifted by prediction market incentives. Feel free to reply to this post with how your behavior was shifted so I can update.
I was hopeful that people might shift their behavior based on the prediction market predictions - specifically that the high probability placed on the home page being blown up would lead to design changes. However, this retrospective clarifies that Petrov Day 2022 was a social experiment, so the prediction would have just shown it was expected to work as designed.
... the primary design of the exercises – seeing how long it'd take for the site to go down, even if we were pretty sure that it would.
It's also possible that people saw the high probability that the home page would be blown up and altered their plans. They may have not blown it up because they expected someone else would (especially if they weren't confident in their anonymity being preserved). Or they may have blown it up because "if it's going to happen, it may as well be me". I think there is a higher chance (25%) that this was an effect, but I don't know what the net direction would have been. Again, feel free to reply if you think the prediction markets helped you make decisions.
Overall I think the prediction markets were a positive addition to a negative event. I think the main incentive in play was the possibility of seeing, or not seeing, the LessWrong home page being blown up. And I think we all have a lot to figure out about prediction markets.
To clarify, me and my friend were 100% going to press the button, but we were discouraged by the false alarm. There was no fun at that point, and it made me lose like 1/3 of my total mana. I had to close all my positions to stop the losses, and we went to sleep. When we woke up it was already too late for it to be noteworthy or fun.
This is a follow-up to the Petrov Day commemoration of 2022
We died.
Twice.
Maybe three times.
This is how it went down...
September, 21, 5:43 PM Pacific Time: I write the code that determines whether a user is authorized to launch. I was in a rush because I leaving for EA Global DC.
September 25, 8:00 PM: Petrov Day celebration commences. The button is on the frontpage. Any users with greater than 2,300 karma could press the button.
September 25: 9:05 PM: I am reminded by others at LessWrong that I was supposed to publish the launch code in the comments. Without the code, no one can nuke the site. Understandably, no one spoke up to point out the absence of code – that would be hella suspicious. This reveals that incompetence on occasion might even save you from destruction. But don't bet on it.
September 25, 10:55:04 PM: The frontpage is nuked after 2 hours and 55 minutes. The karma threshold is 2,100 and there are 290 users with that much karma or more.
September 25, 11:51 PM: Oliver Habryka is suspicious that there is a bug in the code and he digs into it, uncovering that indeed, due to an error in the logic, any user with 0 karma (and exactly 0 karma) was already able to launch nukes and destroy the frontpage. An anonymity-preserving database check confirmed that it was a user with 0 karma who had launched the missiles.
September 26, 12:36 AM: The decision is made to revive the frontpage and keep going with the Petrov Day commemoration.
September 26, 10:11 AM: The entire LessWrong website (not just the front page) goes down and is returning 502s. This is because I pushed a bad commit to fix hiding the Petrov Opt-Out checkbox without properly testing it. The site is soon restored to functioning.
September 26, 5:33:02 PM: At this time, the karma threshold for the ability to launch is 200, and 1,504 users were able to bring down the front page.
With two hours and 27 minutes remaining, someone presses the initiation button, enters the launch code, and presses fire. The frontpage goes down.
With the anonymity introduced in 2022, we do not know who pressed the button or why.
September 26, 8:00:00 PM: The Petrov Day commemoration is concluded. The LessWrong frontpage is restored.
What a day. What a day indeed.
I will tell you the symbolic lessons I take from these events, though you are welcome to interpret things how you please.
First, we very successfully commemorated how shoddy engineering by well-intentioned engineers can still lead to the end of the world. In Petrov's case, the shoddy engineering lead to a false alarm that was safely ignored. In our case, we weren't so lucky, and the bad code meant that the site did indeed go down. And in real life, you don't get to decide that wasn't fun and have a do-over.
We could have decided to leave the site down, there would have been some symbolic purity in that, however it seemed more worthwhile to see what would happen under the primary design of the exercises – seeing how long it'd take for the site to go down, even if we were pretty sure that it would.
Second, I want to claim that although the site went down, it in fact revealed that users on LessWrong with more than 300 karma (1,178 of them exist, 335 visited the site on Petrov Day) are not the kind to push buttons for one reason or another. Granted many of them would not have been checking the site, but still. This is a much better outcome than the Manifold Prediction market anticipated:
You can see that for much of the day, the market thought that the site would survived for only ~30% of the day, i.e. 8 hours. In fact, the site survived 21.5 hours or 90% of the day.
I feel pretty good about that! This is even true despite the fact that a button-pusher would remain anonymous this year.
A couple of people expressed that they were very sad that the site went down this year as it means we don't get the symbol of trust we had in previous years. I'm not sure that's the right argument. Given that 335 users with 300+ karma were active on the site on Petrov Day, and the site didn't go down until we got beneath that, you could argue this is most successful Petrov Day yet on LessWrong (in past years, at most 250 people were given codes, and it's not clear they all visited LessWrong even). Plus, as above, this year the 300+ users didn't press the button despite the offer of anonymity.
Of course, this analysis ignores the fact that someone with 300+ karma might have decided to wait until the threshold was much lower than their own score in order to act with more anonymity. It's very hard to rule that out, so perhaps the above analysis is bunk. Or rather, it's far from certain but is evidence in a certain direction.
Something we introduced this year was the ability to opt out of Petrov Day. There are at least two reasons you might want to do so: 1) because you wish to object to the game, or 2) because you want to commit yourself to not pressing the button.
35 users chose to opt out. We cannot be sure of their motivations, but a few users publicly declared their opt-out, several citing that committing to not press the button seemed like the right thing to do. Kudos to them!
Lastly, Zack Stein-Perlman polled LessWrong's users to check what people overall thought about bringing the site down:
As you can see, overwhelmingly, users think one ought not to press the button.
Hear, hear. I agree.
This was great, y'all. Yes, the site went down eventually, but only once the karma required to do was merely 200. From this we can conclude that in gaining more karma than that, one becomes the kind of person who doesn't destroy the world symbolically or otherwise.
Till next time!
Specifically, the function below determines who could launch nukes. The flaw in this function is that users with 0 karma do not have a
karmafield on their user object, so checking their karma is below the current threshold does nothing, i.e. nothing in the function prevents them from launching.