If you must find a single winner, allow people to change their guess after hearing the later ones, continuing until everyone is happy with their guess (and implicit range). Thus, if player 1 was thinking "somewhere around 50" they can now pick between "exactly 50" "lower" or "higher;" and there will be an auction to determine how far below 50 someone stops (or at what point the "51" player drops down, and maybe gets "overcut"

You could also use something like the "name that tune" algorithm:

1. The first guesser picks a range of values
2. The second guesser picks a range of values that either A. Does not overlap with the first guesses OR B. Covers a different number of values as the first guesser
3. Each subsequent guesser has the same options A. Make a guess that doesn't overlap anyone B. Make a guess with a different number of values as anyone who is overlapped
4. Allow people to revise their ranges, as above.

Then, whoever has the smallest range that contains the true value wins.

The problem with the existing protocol is that it forces the choice of a single winner.  If multiple players are all basically right, the protocol you describe forces them into a deathmatch because only one player can be "the winner".

(Another problem with the existing protocol is that it has some players making their predictions "before" others, in a way that is visible to the others.)

Here's a better protocol: everyone makes their prediction at the same time without seeing anyone else's prediction.  If someone is off by X units then their score for that round is 1/(X+1).  For best results, play several rounds and compute the average score.

You might also be interested in Wits And Wagers, which is the "everyone predicts a number" activity made into a six-player board game.  I've played it.  It's pretty fun.

It seems like the most straightforward simple fix is for everyone to guess simultaneously. Then people can't just grab big ranges because they are big. E.g. the player who guesses 49 in your example would have to have actually had a lower estimate for the value rather than merely cutting you off at the knees. I haven't studied in depth but I think the incentives (if each person is trying to account what the others, N-layers deep) are to just guess your true prediction.

When played as a game, the purpose is rarely for the organizer to get a good estimate.   It's either for entertainment or for point-scoring by the participants.  As such, the bracketing strategy is just part of the fun.  If you want to "fix" it, just make it simultaneous - you get better information about independent estimates, and you remove (some of) the strategic adjustment of guesses.

You could also give up the "one winner" idea - scoring based on distance from actual (or better, by impact of error, which can vary by topic), without caring whether someone else was closer, would incent true estimate submission.

Finally, why not just set up a betting market?  Let participants weight their estimates by strength of information.

 

If you're just doing this occasionally without recordkeeping, then it seems convenient to have the game result in "winners" rather than a more fine-grained score. But it could be fine to sometimes have multiple winners, or zero winners. Here's a simple protocol that does that:

The person who asks the question also defines what counts as "winning". e.g. "What's the value of such-and-such? Can anybody get it within 10%?" Then everyone guesses simultaneously, and all the people whose guesses are within 10% of the true value are "winners".

("Simultaneous" guessing can mean that first everyone comes up with their guess in their head, and then they take turns saying them out loud while on the honor system to not change their guess.)

Slightly more complicated, the asker could propose 2 standards of winning. "When did X happen? Grand prize if you guess the exact year, honorable mention if you get it within 5 years." Then if anyone guesses the exact year they're the big winner(s) and the people who get it within 5 years get the lesser glow of "honorable mention". And if no one guesses the exact year then the people who get it within 5 years feel more like winners.

If you continue farther in this direction you could get to one of Ericf's proposals. I think my version has lower barriers to entry, while Ericf's version could work better among people who use it regularly.

Get each player to assign a probability distribution over answers. Starting with an equal prior over players answers, update it based on the observation. Sample from the posterior. (So if Alice assigned twice as much prob as Bob to the correct outcome, then Alice is twice as likely to win. )

Or, we could give confidence intervals. But how should they be scored?

A relatively simply way to do this would be rather than CI, you give "location and scale" and are scored according to:

Here is a proposal. Not sure if I like it better or worse than my other one.

In all cases, add infinitesimal noise to everyones votes, and the final answer, to avoid ties.

If there are an odd number $2n+1\ge 3$ of people, have everyone submit a $\frac{n}{2n+1}-\frac{n+1}{2n+1}$ confidence interval. Alternate (choosing randomly which to do first) taking the unlabeled person with the lowest remaining lower bound, and labeling them "below", and taking the unlabeled person with the highest remaining upper bound, and labeling them "above." When one person remains unlabeled, they are given the region $(r_0,r_1)$, where $r_0$ is the greatest lower bound among people labeled "below," and $r_1$ is the least upper bound among people labeled "above." Recursively distribute the remaining region less than $r_0$ across people labeled below and the remaining region greater than $r_1$ across people labeled above.

If there are an even number $2n\ge 2$ of people, have everyone name their median guess, let $r$ be the average of the two inner-most guesses, and recursively distribute the region less than $r$ to the people who gave guesses less than $r$, and the region greater than $r$ to the people who gave guesses greater than $r$. 

If there is 1 person, they get the entire region.

This is my initial proposal, but I am going to think about it more:

There are $n$ people. Everyone privately names a number, where the prompt is "What number do you think the answer has probability $1/n$ of being below." Lowest bidder gets the territory $(-\infty ,b_1)$, where $b_1$ is the second lowest bid. 

The remaining $n-1$ people divide up the $[b_1,\infty )$ using the same process, with the prompt "What number do you think the answer has probability $1/(n-1)$ of being below conditioned on being at least $b_1$. Lowest bidder getting the territory $[b_1,b_2)$, where $b_2$ is the second lowest bid.

Continue this pattern until there is only one person left, who gets the remaining region.

This is not symmetric with respect to negating the answers, but it does have the property that everyone has a strategy (honest reporting) that guarantees that they win with subjective probability at least $1/n$. 

Everyone has to name a guess (mean) and range (standard deviation) of a normal distribution. Whoever's pdf takes the largest value at the true answer wins. Bonus: you may opt to invertibly transform the input first in one of several acceptable ways, most notably taking the logarithm. Now take that and simplify it. ;)

1. Everyone guesses a number (could be sequentially or simultaneously, but duplicate guesses are allowed).

2. Anyone who guessed the correct value in the previous step wins.

3. If no guesses were exactly correct, everyone guesses a new number and repeat until one or more players guess correctly.

This only works for reasonably small discrete ranges. For continuous or very large ranges, modify step 2 so that anyone who guesses within some distance of the true value wins.

Is this for a one-shot game or are you doing this over many iterations with players getting some number of points each round?

One simple method (if you are doing multiple rounds) is to rank players each round (Closest=1st, Second Closest=2nd, etc) and assign points as follows:

Points = Number of Players - Rank

So say there are 3 players who guess as follows:

Player 1 guesses 50

Player 2 guesses 49

Player 3 guesses 51

And say the actual number is 52.

So their ranks for that round would be:

Player 1: 2nd place (Rank 2)

Player 2: 3rd place (Rank 3)

Player 3: 1st place (Rank 1)

And their scores would be:

Player 1: 3 - 2 = 1 point

Player 2: 3 - 3 = 0 points

Player 3: 3 - 1 = 2 points

I think this works better if you are calculating a winner over many rounds, so that there is a new ranking and new awarding of points on each round.  The same is true of least squared error, which you mention, and most of the other methods of incentivizing players to try to guess the mean expected value.

I could also think of other ways to incentivize this, and to use confidence intervals, but they all add complexity to the points calculations.