"What's the worst that can happen?" goes the optimistic saying. It's probably a bad question to ask anyone with a creative imagination. Let's consider the problem on an individual level: it's not really the worst that can happen, but would nonetheless be fairly bad, if you were horribly tortured for a number of years. This is one of the worse things that can realistically happen to one person in today's world.
What's the least bad, bad thing that can happen? Well, suppose a dust speck floated into your eye and irritated it just a little, for a fraction of a second, barely enough to make you notice before you blink and wipe away the dust speck.
For our next ingredient, we need a large number. Let's use 3^^^3, written in Knuth's up-arrow notation:
- 3^3 = 27.
- 3^^3 = (3^(3^3)) = 3^27 = 7625597484987.
- 3^^^3 = (3^^(3^^3)) = 3^^7625597484987 = (3^(3^(3^(... 7625597484987 times ...)))).
3^^^3 is an exponential tower of 3s which is 7,625,597,484,987 layers tall. You start with 1; raise 3 to the power of 1 to get 3; raise 3 to the power of 3 to get 27; raise 3 to the power of 27 to get 7625597484987; raise 3 to the power of 7625597484987 to get a number much larger than the number of atoms in the universe, but which could still be written down in base 10, on 100 square kilometers of paper; then raise 3 to that power; and continue until you've exponentiated 7625597484987 times. That's 3^^^3. It's the smallest simple inconceivably huge number I know.
Now here's the moral dilemma. If neither event is going to happen to you personally, but you still had to choose one or the other:
Would you prefer that one person be horribly tortured for fifty years without hope or rest, or that 3^^^3 people get dust specks in their eyes?
I think the answer is obvious. How about you?
Constant, my reference to your quote wasn't aimed at you or your opinions, but rather at the sort of view which declares that the silly calculation is some kind of accepted or coherent moral theory. Sorry if it came off the other way.
Nick, good question. Who says that we have consistent and complete preference orderings? Certainly we don't have them across people (consider social choice theory). Even to say that we have them within individual people is contestable. There's a really interesting literature in philosophy, for example, on the incommensurability of goods. (The best introduction of which I'm aware consists in the essays in Ruth Chang, ed. 1997. Incommensurability, Incomparability, and Practical Reason Cambridge: Harvard University Press.)
That being said, it might be possible to have complete and consistent preference orderings with qualitative differences between kinds of pain, such that any amount of torture is worse than any amount of dust-speck-in-eye. And there are even utilitarian theories that incorporate that sort of difference. (See chapter 2 of John Stuart Mill's Utilitarianism, where he argues that intellectual pleasures are qualitatively superior to more base kinds. Many indeed interpret that chapter to suggest that any amount of an intellectual pleasure outweighs any amount of drinking, sex, chocolate, etc.) Which just goes to show that even utilitarians might not find the torture choice "obvious," if they deny b) like Mill.