From my internal view I'm sceptical of it because I'm familiar with it :-/

You're right, I was assuming things about you I shouldn't have.

Um, hiring from top colleges is not quite all signaling. There is quite a gap between, say, an average Stanford undergrad and an average undergrad of some small backwater college.

Fair point. But the point is that they're going on something like "the average undergrad" and discounting all the outliers. Especially problematic in this case because forecasting is an orthogonal skillset to what it takes to get into a top college.

With people who claim to have awesome forecasting power or techniques, I tend to point at financial markets and ask why aren't they filthy rich.

Markets are one of the best forecasting tools we have, so beating them is hard. But using the market to get these types of questions answered is hard (liquidity issues in prediction markets) so another technique is needed.

Um, I was one of Tetlocks' forecasters for a year. I wasn't terribly impressed, though. I think it's a bit premature to declare that they "solved the problem".

What part specifically of that paper do you think was unimpressive?

discounting all the outliers

Not necessarily. Recall that a slight shift in the mean of a normal distribution (e.g. IQ scores) results in strong domination in the tails.

Besides, searching for talent has costs. You're much better off searching for talent at top tier schools than at no-name colleges hoping for a hidden gem.

using the market to get these types of questions answered is hard

What "types of questions" do you have in mind? And wouldn't liquidity issues be fixed just by popularity?

forecasting is an orthogonal skillset to what it ta

... (read more)

Open Thread, Dec. 28 - Jan. 3, 2016

by [anonymous] 1 min read27th Dec 2015145 comments


If it's worth saying, but not worth its own post (even in Discussion), then it goes here.

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should be posted in Discussion, and not Main.

4. Open Threads should start on Monday, and end on Sunday.