I don't have all the specific knowledge about AGI, but like any living being, I'm interested in my own survival, if possible without killing others (it's one of the defining characteristics of human beings). I'm not a native English speaker, hence the potential for misunderstandings.
"I mean that in practice, using the techniques we actually have, "please don't disassemble literally everyone with probability roughly 1" is an overly large ask that we are not on course to get. So far as I'm concerned, if you can get a powerful AGI that carries out some pivot... (read more)
To those who gave my first post a negative rating, a negative rating isn't an explanation, it's a personal opinion. Nature (or God) took 6? 10? million years to eliminate bad genes, or at least to find the optimal way to use them. As I pointed out, you're not being tested for ALL diseases, including non-genetic ones, just for a few, which doesn't put you in real-life situations. In short, you'll probably have some disappointments. Sorry to burst your bubble...
Don't be afraid to say 'oops' and change your mind.