2025 Prediction Thread
14Nick_Tarleton
1Ben Pace
4Nick_Tarleton
7jimrandomh
14Ben Pace
4jimrandomh
4Ben Pace
5Ben Pace
2CstineSublime
3gw
1winstonBosan
1gw
2Mateusz Bagiński
2isabel
7habryka
3jimrandomh
4Screwtape
2jimrandomh
1CstineSublime
1Stephen McAleese
1ashtree
New Comment
How can I remove an estimate I created with an accidental click? (Said accidental click is easy to make on mobile, especially because the way reactions work there has habituated me to tapping to reveal hidden information and not expecting doing so to perform an action.)
This doesn't work. (Recording is Linux Firefox; same thing happens in Android Chrome.)
An error is logged when I click a second time (and not when I click on a different probability):
[GraphQL error]: Message: null value in column "prediction" of relation "ElicitQuestionPredictions" violates not-null constraint, Location: line 2, col 3, Path: MakeElicitPrediction instrument.ts:129:35
1
1Sorry about that, a fix is in progress. Unmaking a prediction will no longer crash. The UI will incorrectly display the cancelled prediction in the leftmost bucket; that will be fixed in a few minutes without you needing to re-do any predictions.
1FWIW I would prefer to have to click to show the distribution, so I could vote before being anchored.
You can change this in your user settings! It's in the Site Customization section; it's labelled "Hide other users' Elicit predictions until I have predicted myself". (Our Claims feature is no longer linked to Elicit, but this setting carries over from back when it was.)
2Bug report: It does have the amusing property that, if you hover over a part of the claim where others have left votes, the text underneath vanishes. Normally it would be replaced with the names of the users who voted, but now it shows no text. This doesn't reveal the key identity bits, but does reveal non-zero bits about what others think.
For data, here are annual counts of quick takes / shortforms.
- 2018: 85
- 2019: 634
- 2020: 887
- 2021: 1028
- 2022: 1258
- 2023: 1580
- 2024: 2286
I'm somewhat surprised to see the distribution of predictions for 75% on FrontierMath. Does anyone want to bet money on this, at say, 2:1 odds (my two dollars that this won't happen against your one that it will)?
(Edit: I guess the wording doesn’t exclude something like AlphaProof, which I wasn’t considering. I think I might bet 1:1 odds if systems targeted at math are included, as opposed to general purpose models?)
Is the bet for general purpose model still open? I guess it depends on the specific resolver/resolution criteria - considering that OpenAI have gotten the answer and solution to most of the hard questions. Does o3's 25% even count?
Yeah I think I would still make this bet. I think I would still count o3's 25% for the purposes of such a bet.
oh, I like this feature a lot!
what's the plan for how scoring / checking resolved predictions will work?
I don't think I would want any kind of site-wide scoring, since I explicitly don't want people to have to worry too much about operationalization.
I do think if people keep using the feature, we'll add some way to resolve things, and resolved markets should have some way of showing whose predictions were right, and send people notifications. And maybe we'll also show people's predictions on their profiles at some point.
If the thing really takes off, of course there is a whole world of integrating prediction things more deeply into LW, but that seems somewhat unlikely given Manifold's existence and success.
1We don't have any plans yet; we might circle back in a year and build a leaderboard, or we might not. (It's also possible for third-parties to do that with our API). If we do anything like that, I promise the scoring will be incentive-compatible.
. . . Okay, I'll bite.
Edit: And-
Now, I don't suppose that LessWrong prediction API is documented anywhere?Kinda. There's source code here and you can poke around the API in graphiql. (We don't promise not to change things without warning.) When you get the HTML content of a post/comment it will contain elements that look like <div data-elicit-id="tYHTHHcAdR4W4XzHC">Prediction</div> (the attribute name is a holdover from when we had an offsite integration with Elicit). For example, your prediction "Somebody (possibly Screwtape) builds an integration between Fatebook.io and the LessWrong prediction UI by the end of July 2025" has ID tYHTHHcAdR4W4XzHC. A graphql query to get the results:
query GetPrediction {
ElicitBlockData(questionId:"tYHTHHcAdR4W4XzHC") {
_id predictions {
createdAt
creator { displayName }
}
}
}Meta question for those who have made predictions: How do go about making a prediction? As in What is your prediction making process?
Which I suppose this is really a melange of questions that decomposes into:
Which questions appealed to you as being worth predicting, and why?
How did you determine what specific conditions the question was asking you to make a prediction about?
What was your process for determining your own level of confidence in that state of affairs?
Is the process similar or dissimilar from how you go about making decisions with tangible effects in your personal, familial, and professional life?
One prediction I'm interested in that's related to o3 is how long until an AI achieves superhuman ELO on Codeforces.
OpenAI claims that o3 achieved a Codeforces ELO of 2727 which is 99.9th percentile but the best human competitor in the world right now has an ELO of 3985. If an AI could achieve an ELO of 4000 or more, an AI would then be the best entity in the world at competitive programming and that would be the "AlphaGo" moment for the field.
An AI system replicating itself seems very unlikely because AI labs are hopefully and presumably protecting against that in particular. However, there are many other dangerous things an AI system could do that aren't self-replication and are often worse. It also seems that if that does happen, we are doomed since AI labs are trying their hardest to prevent that, and if they fail to prevent that, we have a self-replicating non-aligned AI, and so we are screwed.