One of the more interesting papers at this year's AGI-12 conference was Finton Costello's Noisy Reasoners. I think it will be of interest to Less Wrong:
This paper examines reasoning under uncertainty in the case where the AI reasoning mechanism is itself subject to random error or noise in its own processes. The main result is a demonstration that systematic, directed biases naturally arise if there is random noise in a reasoning process that follows the normative rules of probability theory. A number of reliable errors in human reasoning under uncertainty can be explained as the consequence of these systematic biases due to noise. Since AI systems are subject to noise, we should expect to see the same biases and errors in AI reasoning systems based on probability theory.
The worst that noise can do is decrease the quality of the approximation that the AI is using. (EDIT: barring the OP effects, of which I am skeptical) For friendliness, this means decreasing decision quality.
If you decide that such is unacceptable, the AI needs to spend more resources (time and energy) on coming to the conclusion. In some cases that will be worth it, in others, not. The AI is capable of making this trade off on it's own.
If you don't let it trade accuracy for speed, the day will come when you need a decision now, and the AI will choke and e... (read more)
One of the more interesting papers at this year's AGI-12 conference was Finton Costello's Noisy Reasoners. I think it will be of interest to Less Wrong: