This is a linkpost for https://federicorcassarino.substack.com/p/super-forecasters-as-a-service
A prediction by a good predictor that's made in one minute isn't as valuable as one made by spending a few hours of research. "What will be my income in 2025 if I switch to X career?" is a question for which making a good forcast not only needs forcasting skills but also research or a lot of existing knowledge.
The ranked list doesn't tell you whether or not someone is a superforcasters. Metaculus could tell you but they don't offer that kind of information publically. They use predictive skill to calculate their metaculus prediction in a way that's completely separate from the ranking score.
Superforcasting as a service also exists but more in a consulting model that sells a higher quality product for a higher price.
Isn't singling out individual market participants as 'superforecasters', and privileging their individual predictions based on past performance, missing the whole point of prediction markets?
Yes. I would still use it though. For grand-scheme-of-things stuff, Metaculus is great. For stuff that's personally relevant (What will be my income in 2025 if I switch to X career?) predictions from people with great track records is good enough for me.
Note that the title is super-forecasters as a service, not prediction-markets as a service.
I've just started building a website yesterday that I think would be super interesting, but I'm not sure if it's legal or if Metaculus/Elicit would be fine with it.
The idea: Super-forecasters as a Service.
Say I wanted to know if I should book a flight to Costa Rica this summer, but I'm hesitant because of Covid flight restriction uncertainty. I could create the following question using Elicit: "Will commercial flight BA2490 from the UK to Costa Rica in July 24 be cancelled?"
The website would let you embed your elicit question, and pay people for predicting. People get paid based on their Points-per-question in Metaculus. You can "auction" 100 dollars on a question, and pick a base price per prediction multiplied by the Points-per-question metric.
The more money you auction, the more predictions you'll get. The higher the multiplier, the more predictions will tend to come from top-predictors.
Benefits:
Drawbacks:
Questions:
Originally posted to Substack: https://federicorcassarino.substack.com