In this post, I give my rules of thumb for dealing with infohazards, which are as far as I can tell wildly divergent from the accepted views of the larger alignment community. I wrote this post less as a means of propagating my own views, and more because I'm curious as to why my views are so uncommon.
My Infohazard Rules of Thumb
Two people can keep a secret if one of them is dead (seems pretty uncontroversial)
Nothing can be alllll that dangerous if it's known to literally everyone how it works (this is probably the crux where I differ wildly from the larger alignment community, if I had to guess)
By the first rule, the only stable numbers of people that can know a secret are zero, one, and literally everyone. (Seems like a pretty straightforward inference, although there's a difference between "permanently stable" and "stable for a long time" that I would guess is probably more salient to other people than it is to me.)
By the second rule, having a secret known to n > 1 people but not literally everyone is basically just a power play that doesn't make anyone safer. It's just a question of people trying to secure a first mover advantage so that they have more bananas and get to sit higher on the tree and boss the other monkeys around once we inevitably get to the stable state that the secret is known to everyone.
Therefore, by the first four rules, if you know a supposed infohazard, and you know or strongly suspect that it is known to at least one other person, then you are morally obligated to broadcast the infohazard after some sort of responsible disclosure process with your local government.
Anyone following the above rules of thumb will thus act in the opposite manner to anyone who buys into the traditional infohazard framework, making them incredibly difficult to coordinate with. Let's call the people who follow the traditional infohazard model, infohoarders, and let's call the people who follow the mental models I am describing here infopandoras.
How can a community work best if it is a mixture of infohoarders and infopandoras? Clearly the infohoarders will want to keep all infohazards away from any infopandora; just as clearly, infopandoras will not declare themselves to be such if they want to attain any position of influence in a community dominated by infohoarders. This seems like a recipe for some seriously unproductive dynamics in trying to solve coordination problems, which are always frequent in life, and generally the hardest problems for humans to solve.
It also seems possible to me that infohoarders will naturally and inevitably dominate any community that values knowledge for its own sake. Infohoarders are like black holes for infohazards, always growing larger and more hazardous over time. Infopandoras are something else, maybe like a supernova, creating one huge blast of infohazard which then settles down to become a new galaxy or what have you after the chaos subsides.
Acknowledgments
I kind of expect this post to be wildly unpopular, just because of how wildly the models presented diverge from the accepted wisdom in the alignment community. So I struggled with the question of whether to cite the people who influenced this post. Ultimately, since I identify as an infopandora, I thought I would just bite the bullet and cite my main influence, which is @jessicata. I've never met her and she doesn't know I exist, but a lot of this mental framework is derived from reading her public writings and just kind of pondering the related questions for a few years.
And, to maybe put too fine a point on it, literal DIY viral gain-of-function research is well within the grasp of crazy ideologues and always has been.
Don't wash your hands after taking a dump. Don't wear a mask to large public gatherings during COVID season. Don't vaccinate your children against dangerous/deadly diseases that were previously reduced to nuisance levels. Have unprotected sex with people you just met. Eat random animals that have enough homologous proteins to act as animal reservoirs for nasty diseases, while making sure to confine those animal in horrible overcrowded conditions that make sure that they do act as animal reservoirs. Overuse antibiotics literally every chance you get.
Is the CDC supposed to suppress this information? Of course not, their whole mission in life is to spread all the information in the above paragraph to anybody who will listen to it. And, at least in the case of COVID, they are doing and have done a piss-poor job of it. Should the rationalist community have suppressed all knowledge of COVID between December 2019 and March 2020? Or is it a badge of honor for the community that they/we were the only major intellectual community to react with the appropriate level of paranoia and infopandoraing that I am advocating in this post and comments?
In this post, I give my rules of thumb for dealing with infohazards, which are as far as I can tell wildly divergent from the accepted views of the larger alignment community. I wrote this post less as a means of propagating my own views, and more because I'm curious as to why my views are so uncommon.
My Infohazard Rules of Thumb
Acknowledgments
I kind of expect this post to be wildly unpopular, just because of how wildly the models presented diverge from the accepted wisdom in the alignment community. So I struggled with the question of whether to cite the people who influenced this post. Ultimately, since I identify as an infopandora, I thought I would just bite the bullet and cite my main influence, which is @jessicata. I've never met her and she doesn't know I exist, but a lot of this mental framework is derived from reading her public writings and just kind of pondering the related questions for a few years.