The OP framed the scenario in terms of directing the AI to design a FAI, but the technique is more general. It's possibly safe for all problems with a verifiable solution.
Good idea. But this effectively makes failing to "go quietly" punishable by death.
In some attacks it's okay to hold of on proposing solutions. In others, it's not. Presumably, there actually are some bad people in Azkaban, and not just, say, people responsible for an accidental death. Before Harry destroys the prison, he needs to think carefully about what is to become of these people.
What's required of a maximum security wizard prison? You clearly need to subdue any magical powers which would allow the prisoners to revolt or escape. At a minimum then, confiscate wands and put up anti-Disapparition charms. This might not be enough, as in canon it's possible to perform magic without a wand. Voldemort was able to terrorize the other orphans by... (read more)
Transitivity? In The Lifespan Dilemma, Eliezer presents a sequence (L_n) in which we are convinced L_n { L_(n+1) throughout, but for which we'd prefer even L_0 to L_n for some large but finite n.
Chapters 55-58 seemed to me to contained very little content. At least not much that was fun/interesting. What content they had was superfluous and repetitive. The only real obstacle for Harry were the Dementors*, and he seemed to defeat them trivially. At the end of Ch. 54, suspense was high, but (at least from my perspective) it really fizzled out.
They're finally out of there. Let us never speak of these chapters again!
Strong recursion: Software designs new software to design newer software; money begets money begets more money. Think of the foom as compound interest on intelligence.
Suppose A designs B, which then designs C. Why does it follow that C is more capable than B (logically, disregarding any hardware advances made between B and C)? Alternatively, why couldn't A have designed C initially?
Oh... I in no way endorse the above argument! Pierre-Simon Laplace's, a century or so after Newton, gave a naturalistic model of how the Solar System could have developed. "Rationality quotes" is not only about sharing words of wisdom, but also words of folly.
Numberplates?
There was recently a submission here posing criticism a well-known contributor, Eliezer Yudkowsky. Admittedly, whatever question the poster intended to ask was embedded within a post which was evidently designed to be a test of our rationality. Eliezer didn't seem to care too much, stating that the issue was
it being a top-level post instead of Open Thread comment. Probably would've been a lot more forgiving if it'd been an Open Thread comment. . .
The question would, under other circumstances, be just - but I don't care to justify myself here. Elsewhere, perhaps. . .
This belongs as a comment on the SIAI blog, not a post on Less Wrong.
I asked Eliezer why mormon2's... (read 529 more words →)
Can the question given in this post be formulated precisely?
If so, nix everything but the precise description of the answer-box's behavior and ask for a program which simulates such a device.
If not, ... then I choose to interpret it in such a way that I can ask for the above anyway.