The OP framed the scenario in terms of directing the AI to design a FAI, but the technique is more general. It's possibly safe for all problems with a verifiable solution.
Good idea. But this effectively makes failing to "go quietly" punishable by death.
In some attacks it's okay to hold of on proposing solutions. In others, it's not. Presumably, there actually are some bad people in Azkaban, and not just, say, people responsible for an accidental death. Before Harry destroys the prison, he needs to think carefully about what is to become of these people.
What's required of a maximum security wizard prison? You clearly need to subdue any magical powers which would allow the prisoners to revolt or escape. At a minimum then, confiscate wands and put up anti-Disapparition charms. This might not be enough, as in canon it's possible to perform magic without a wand. Voldemort was able to terrorize the other orphans by "hand-to-hand" magical means before he had even been introduced to wand-based magic! So what else can you do? You could Somnium prisoners for the duration of their sentence, but this seems both inhumane and ineffective as a means of punishment (if that's actually a goal of imprisonment).
We don't really know enough to say to what degree it's possible to subdue a wizard's magical powers without bringing in the Dementors. If Dementors are just a reification of the fear of death, perhaps you could terrorize the prisoners in some manner as to achieve a similar effect. This would be unacceptable from Harry's point of view. In canon, we see that some people's magical abilities diminish due to heartbreak (Tonks and possibly Tom Riddle's mother). It might be possible to exploit this phenomenon somehow, but then again it could fail to work on psychopathic prisoners.
Given what we don't know, it's possible that all solutions to this problem are inhumane (i.e. the only way to suppress magical ability is by trauma). We don't live in the HP:MoR universe, so we can't do much research on the possibilities, but Harry should!
EDIT: The thought occurred that you could transfigure the prisoners into Muggles. Could be possible, but only for short periods of time (and may require at least a 1-1 ratio of guards to prisoners).
Transitivity? In The Lifespan Dilemma, Eliezer presents a sequence (L_n) in which we are convinced L_n { L_(n+1) throughout, but for which we'd prefer even L_0 to L_n for some large but finite n.
Chapters 55-58 seemed to me to contained very little content. At least not much that was fun/interesting. What content they had was superfluous and repetitive. The only real obstacle for Harry were the Dementors*, and he seemed to defeat them trivially. At the end of Ch. 54, suspense was high, but (at least from my perspective) it really fizzled out.
They're finally out of there. Let us never speak of these chapters again!
Strong recursion: Software designs new software to design newer software; money begets money begets more money. Think of the foom as compound interest on intelligence.
Suppose A designs B, which then designs C. Why does it follow that C is more capable than B (logically, disregarding any hardware advances made between B and C)? Alternatively, why couldn't A have designed C initially?
Oh... I in no way endorse the above argument! Pierre-Simon Laplace's, a century or so after Newton, gave a naturalistic model of how the Solar System could have developed. "Rationality quotes" is not only about sharing words of wisdom, but also words of folly.
Numberplates?
Can the question given in this post be formulated precisely?
If so, nix everything but the precise description of the answer-box's behavior and ask for a program which simulates such a device.
If not, ... then I choose to interpret it in such a way that I can ask for the above anyway.