To address your question more directly, Eliezer thinks that it should be possible to create accurate but nonsentient models of humans (which an FAI would use to avoid simulating people in its environment as it tries to predict the consequences of its actions). This seems plausible, but if you attach a microphone and speaker to such a model then it would say that it's conscious even though it's not. It also seems plausible that in an attempt to optimize uploads for computational efficiency (in order to fit more people into the universe), we could unintentionally change them from sentient beings to nonsentient models. Does this convince you that it's not safe to "just ask an upload whether it's conscious"?
Optimizing an upload can turn it into Eliza! Nice :-)
I still think that if a neuron-by-neuron upload told you it was conscious, that would probably mean computer programs can be conscious. But now I'm less sure of that, because scanning can be viewed as optimizing. For example, if the neuron saying "I'm conscious" gets its input from somewhere that isn't scanned, the scanner might output a neuron that always says "yes". Thanks for helping me realize that!
.