https://slatestarcodex.com/2013/06/14/the-virtue-of-silence/
Altruistic silence is probably my default position, but from a strictly rational standpoint, is there some way to get paid for my continued silence (other than with the joy of living in a world ignorant of this idea)?
This belies a misunderstanding of what 'rational' means. Rational does not mean homo economicus, it means doing what a person would actually want to do on reflection if they had a good understanding of their options.
I doubt your idea is actually that dangerous, so I'm treating this as a hypothetical. But in general if your idea is dangerous and you want hush money to keep silent about it then this is really more like a blackmail threat than anything else. I think you should reflect on what life decisions you've made that posting what amounts to a "it'd be a real shame if..." threat on a public forum seems like a good idea.
And while you're at it, delete this.
I am holding a lot of dangerous knowledge and am encumbered by a variety of laws and non-disclosure agreements. This is not actually uncommon. So arguably, I am already being paid to keep my mouth shut about a variety of things, but these are mostly not original thoughts. This specific idea is, in my best judgement, both dangerous, and unencumbered by those laws and NDAs.
The assertion that my default position is 'altrusitic silence' means that this is not 'posting a threat on a public forum'. It would be a real shame if a large variety of things that
I doubt your idea is actually that dangerous, so I'm treating this as a hypothetical. But in general if your idea is dangerous and you want hush money to keep silent about it then this is really more like a blackmail threat than anything else. I think you should reflect on what life decisions you've made that posting what amounts to a "it'd be a real shame if..." threat on a public forum seems like a good idea.
I think this a really unfair reading of this post. Maybe it has been edited in ways since it was originally posted to chang...
Here's some good thoughts on dealing with dangerous ideas:
Thank you for this, it is the type of response I was looking for, and I now have a new blog to read regularly.
Unfortunately, my only motivation for staying quiet here is altruism, for others in similar circumstances, how could they be incentivized by the world at large to avoid disclosure of unambiguously harmful ideas?
I don't think we have a general solution to this problem, but it would be great if we could find one because I believe it to be equivalent to solving the coordination problems we face in avoiding unsafe superintelligent AI (assuming you believe superintelligent AI is unsafe by default, as I do). You might find some insight towards mechanism people have considered by looking at posts here about AI policy related to avoiding and preventing the development of unsafe AI.
I think I have a general solution. It requires altruism, but at the social rather than the individual level.
This concept of a reverse patent office as described is based on the idea that if a toxic meme emerges, it would be more harmful if it had emerged in the past, and that any delay is good.
The reverse patent office accepts encrypted submissions from idea generators, and observes new ideas in the world.
When the reverse patent office observes a harmful idea in existence in the world, and assesses it as worth 'having been delayed' based on pre-existing c
Thank you! I was hoping that someone was aware of some clever solution to this problem.
I believe that AI is at least as inherently unsafe as HI, 'Human Intelligence'. I do think that our track record with managing the dangers of HI is pretty good, in that we are still here, which gives me hope for AI safety.
I wonder how humans would react to a superintelligent AI stating 'I have developed an idea harmful to humans, and I am incentivized to publicize that idea. I don't want to do harm to humans, can you please take a look at my incentives and tell me if
How should I dispose of a dangerous idea?
To "dispose of an idea" requires more than just not acting on it yourself or not sharing it with others. You also have to make sure other people don't have it.
from a strictly rational [Economic] standpoint, is there some way to get paid for my continued silence
While you may be confident no one else would think of it, those of us who (supposedly) don't know it, may find it a little harder to buy that you have a monopoly.
This is a sufficiently niche area, that you might need to generalize more - preventing catastrophes/evil... whatever this is.
You could create or try to find a list of ideas that are already out there in the world (that seem dangerous).
Like: https://en.wikipedia.org/wiki/Assassination_market already exists, and if that was your idea, then the 'secret' is already out.
True! Unfortunately, I do think that in this particular case, it is both unique and undiscovered, as it is a very weird synthesis of unrelated fields, and I can say with some confidence that the list of people with knowledge and experience equalling or exceeding mine in all of the relevant disciplines is likely small enough that one of us would have to be 'first', and in this case, it seems to be me.
If it were as widely known as, for example, the idea of the assassination market, it would already have been used, the fact that it has not been discussed in any of the relevant fields, and has not been implemented, suggests that it is in fact novel.
This is an appeal to my expertise and relies on mystique, so I wouldn't find it very persuasive, unfortunately.
So you are stating that you are the only person you know of who has this dangerous idea, even after looking online (carefully, to avoid accidentally giving it away)? Unless you are in a very special situation, I find it hard to believe that anything unique you know, if disclosed, can measurably change the world to the worse. My estimate of this eventuality is probably at the noise level, somewhere at the level of a non-physicist purporting to have solved one of the mysteries of fundamental physics.
To say that special circumstances exist, and that my research was thorough would be an unpersuasive appeal to my own authority.
I assert that this is closer to a professional engineer noticing a new, non-obvious application of a technology that he is well acquainted with. This happens daily and in theory is required in order to receive a patent.
This idea prompted the desire for a 'reverse patent', where someone who generates a strictly harmful idea is somehow economically incentivized to avoid disclosure. Unfortunately, disclosure to a 'reverse patent office' would still be disclosure, and therefore harmful.
If the downvotes and comments are any indication, the community that is concerned with the idea of an 'artificial' intelligence coming up with some unanticipated engineering breakthrough that harms humanity, then accidentally or intentionally turning it loose is pretty hostile to a 'natural' intelligence asserting that it has done the same, looking for a way other than altruism to motivate others in the same position to keep quiet.
You got downvotes because you made a blackmail thread against the community. Even if it's no credible threat, treating it as an hostile act and downvoting it is still right for game theoretic reasons.
So in your read of the downvotes, the most common interpretation of the OP was 'you community members should pay me, an outsider, to be virtuous, or else' rather than 'hi fellow rationalists, does anyone know of resources that would allow me to profit from the practice of our shared values'?
I have a dangerous idea which, unlike most of my dangerous ideas, is unencumbered by any laws or non-disclosure agreements, so I can dispose of it as I wish.
In order to demonstrate that I do in fact have a dangerous idea, I would obviously have to disclose it, but disclosure would not prompt the development of countermeasures other than 'wow, hope nobody else thinks of that'. Any implementation of the idea would probably result in its full disclosure to the world, meaning, no entity could both implement the idea, and keep enough of the idea a secret to prevent others from using it.
I believe implementation of the idea by myself to be immoral, but am conflicted on the ethics of simple disclosure, which I believe would lead to its' implementation by others with more flexible ethics.
Altruistic silence is probably my default position, but from a strictly rational standpoint, is there some way to get paid for my continued silence (other than with the joy of living in a world ignorant of this idea)?
Edit per request from the comments, I work in an industry related to one of the ones EA has identified as harmful. I view this particular idea as similarly harmful to 'a proposal for a marketing campaign that would be substantially more effective than existing marketing campaigns at convicing children and teenagers to start using nicotine' (I do not work in the nicotine industry, and the analogy is imperfect because I think that countermeasures could be developed to a marketing tactic). As my subfield is arcane enough that there are not many people likely to run across my own idea, and if I keep quiet (which I intend to do), there is a good chance that it will not be discovered by someone else.
Unfortunately, my only motivation for staying quiet here is altruism, for others in similar circumstances, how could they be incentivized by the world at large to avoid disclosure of unambiguously harmful ideas?