I've made a couple of comments on the text. This is very centrally the kind of work I do in two ways:
- I work on AI community notes writers and have a lot of contact with the X community notes team. I do not work for X.
- I work on trying to get an epistemic standard (community notes) more widely adopted. I am focused more outside of X but I think there are transferable lessons.
Okay so thoughts, from a variety of angles.
The proposed strategy is expensive. As I say next to the specific quote, anything that changes the whole of X is very expensive. So having a type of tweet called a prediction which automatically applies to all prediction like tweets.. expensive! Gonna require huge amounts of buy in, I guess maybe up to the head of Product (70%). I suggest instead looking at why apps where it tracks predictions have failed. Notably, stuff like Metaforecast.
To my reading (and I tend to have mediocre comprehension when I am answering quickly) This doesn't engage with legitimacy at the time. Whomst among us has not spent time down the resolution criteria mines. It's not just about getting some criteria it's about getting criteria which nail the issue. If people read the criteria for "Is the strait of Hormuz open by October 1st, 2026?" is it gonna match what they think? Well how much more so random tweets. And I think that is something that needs to happen at the time of creation, not just resolution. Ideally the community notes-style process comes in early, nails down what the prediction is and then maybe comes in to check we think we got it right. Personally I'd like to do this for AI 2040. Get people to try and say what they thing the forecasts are now.
I think there is a question of MVP. Like you run lesswrong. Why suggest something on X when it doesn't really exist well here? Why would you expect them to do it better than you?
Happy to have a chat, dm me and we can talk.
Thanks for substantive reply!
Yep, this is a big swing.
This is backchaining from "what could possibly help, for substantively raising the median sanity, or at least median sanity of the intellectual class?". Things that help more "on the margin" are also useful and people should do them but, man, we really need a lot of sanity really fast IMO. I think we will need some big swings.
That said, obviously if you were going to do this, you'd want to start with cheaper ways to prototype it. (i.e. experiment with it on Lesswrong, Manifold, glosso, etc, and initially roll it out as an opt in beta thing on twitter).
But, I want to start by asking "when we simulate the good, smooth version of the product, implemented magically perfectly, does the big swing even seem like it'd work?" and iterate on that conceptually before worrying about the simpler dogfood prototyping.
The big swing needs to engage directly with "what seems incentive compatible with 'most people don't care about truth, really' and "pundits and leaders actively resist submitting to processes that could rule them out" and "any process that has teeth will be a target of politicization".
Whomst among us has not spent time down the resolution criteria mines. It's not just about getting some criteria it's about getting criteria which nail the issue. If people read the criteria for "Is the strait of Hormuz open by October 1st, 2026?" is it gonna match what they think?
This seems to be missing the point of the post, though. The whole point is "resolution criteria is basically a hard blocker on accurate predictions going mainstream. So how do we completely route around it?"
The post is proposing the hypothesis, that to square that circle:
- you make it happen frictionless/automatically on a big platform people already use.
- you make that happen by selling Musk on it (which seems hard but doable? modulo prototyping it in lower stakes places first and, like, you know, make a world-class UI and sort out novel algorithmic problems or whatever, but, "that's the easy part")
- you use LLMs to automate it as much as you can (in particularly aggregating and summarizing the object level evidence).
- you use community notes algorithm to handle "resolve complex predictions quickly without having to deal with resolution criteria, in a way that will hold up."
- use various UI tricks to make it feel like a fun game instead of a punishment.
I think the main thing the post is asking is "does #4 actually work?". It feels intuitively to me like a mix of "LLMs to make an initial ruling and highlight evidence, community notes voting to resolve controversial stuff" should work. But, I'm interested in thoughts from people who have more experience with the gritty details o that.
("is it actually possible to make this a fun game that mainstream people care about, or at least mainstream aspiring intellectuals, if you solve all the other problems?" feels like the second hardest part)
1The big swing needs to engage directly with "what seems incentive compatible with 'most people don't care about truth, really' and "pundits and leaders actively resist submitting to processes that could rule them out" and "any process that has teeth will be a target of politicization".
I guess in that case. I consider myself engaged in that 'big swing'. I am trying to create a norm of community notes and providence tracking on the web, both through my organisation (Goodheart Labs)' work and through a fund I hope to launch soon focused on this work.
Personally I think community notes is a better big swing than forecasting.
But to engage with your theory.
- you make it happen frictionless/automatically on a big platform people already use.
- you make that happen by selling Musk on it (which seems hard but doable? modulo prototyping it in lower stakes places first and, like, you know, make a world-class UI and sort out novel algorithmic problems or whatever, but, "that's the easy part")
- I think this is probably the hardest part
- you use LLMs to automate it as much as you can (in particularly aggregating and summarizing the object level evidence).
- you use community notes algorithm to handle "resolve complex predictions quickly without having to deal with resolution criteria, in a way that will hold up."
- I think this needs to happen at the start, not the end. There should be agreement while the forecast is in flight.
- If you think this, do you think this would work on LessWrong, why why not? (I don't think it would)
- use various UI tricks to make it feel like a fun game instead of a punishment.
Like my sense is that people don't like or aren't capable of back rationalisation on controversial topics to come to conclusions they don't like. But they are if they agreed beforehand. So personally I expect a process (sure, I agree, a community notes-like process) to happen during the forecast.
Is that better?
1Personally I think community notes is a better big swing than forecasting.
Well insofar as community notes is the bedrock that enables this thing, yeah it seems basically "strictly better as a big swing." You'd have to invent it first to get people warmed up to using it in other contexts.
providence tracking on the web,
Could you explain more about what that means?
Providence tracking is about knowing where images and videos were first seen. It is very useful as a way of trivially knowing that some claims backed up by images are inaccurate (because the image existed before the claimed time). Pangram for images is part of this, ie knowing which ones are AI generated.
I think this needs to happen at the start, not the end. There should be agreement while the forecast is in flight.
...
Like my sense is that people don't like or aren't capable of back rationalisation on controversial topics to come to conclusions they don't like. But they are if they agreed beforehand. So personally I expect a process (sure, I agree, a community notes-like process) to happen during the forecast.
Mm, yeah sounds right. I assume here you mean agreement on the parameters/ironing-out-operationalization as much as is practical?
If you think this, do you think this would work on LessWrong, why why not? (I don't think it would)
I feel something like "this is overkill for LessWrong." (We've particularly tried out community-notes algorithms... and, well, for good or for ill, there just doesn't actually seem to be major competing clusters in LW that need to be community-notes-algorithm'd – the results aren't noticeably different from the normal karma system)
I think it'd be good to have various automated nudges on LW towards better epistemics. I think this isn't "the piece that LW is missing" in a way that it feels more like "a piece that twitter is missing." The reason to focus on predictions is because it's a way to inject epistemics into the broader populace that it's easier to justify.
Insofar as I believe this is the right thing for twitter, I do think I should prioritize making some version of it happen on LW and if it's not working here that is evidence it wouldn't work there.
Mm, yeah sounds right. I assume here you mean agreement on the parameters/ironing-out-operationalization as much as is practical?
And just what the rough thing actually means. Like sometimes a question turns out to mean quite different things. We can drill down on the distiction, but I guess there are examples where it isn't res criteria, broadly construed.
I feel something like "this is overkill for LessWrong." (We've particularly tried out community-notes algorithms... and, well, for good or for ill, there just doesn't actually seem to be major competing clusters in LW that need to be community-notes-algorithm'd – the results aren't noticeably different from the normal karma system)
I don't think this is why you don't have forecasts more widely locked/tracked on lesswrong.
Insofar as I believe this is the right thing for twitter, I do think I should prioritize making some version of it happen on LW and if it's not working here that is evidence it wouldn't work there.
I would watch with interest.
1I think a takeaway from this convo is actually "I want to build a singleplayer writing tool that just naturally prompts you to notice prediction-implications of your writing and help operationalize them", and if that seems to be going well is more natural to try out in multiplayer contexts.
I don't think this is why you don't have forecasts more widely locked/tracked on lesswrong.
We've specifically thought about forecasts several times, and each time we end up feeling "idk, the forecasts just don't quite seem to be doing the same type of work that LW posts are doing, like, the fact that you need to do all this annoying operationalization to make them trackable is pretty annoying and doesn't really feel like it's Doing The Thing."
This thread has made me feel more optimistic about routing around that (less with community notes probably, more with AI assistance)
Why do you think that's different than twitter? Like of LW makes claims about the future, right?
When you make a prediction shaped tweet, it automatically gets converted into a prediction-y object*.
This seems a huge ask to me. Like 100x or 1000x more work than some nearby things. Very expensive to do anything that affects most users.
Is there some kind of "get prediction markets, or predictions, onto twitter as a central object" project going? If so, how is it going?
I have been pushing on this for a couple of years. My general take on X (at least as regards the community notes team) is that they do the sort of things I would expect to be good. If I have an idea they tend to implement it eventually.
1On the specific question of "when are prediction markets landing on Twitter", I'd defer to @Nathan Young or Jay Baxter
More broadly, I think AI suggested operationalizations and resolutions are quite promising as a way of reducing human burden of maintaining markets. Manifold does have a built in system for eg pasting in a news article and suggesting markets on the outcome; I'm not sure how often it's used or how good it is though. Occasionally a creator will defer resolution to AI, though that's still quite rare.
(AI also seems like a good first layer for some kind of courtlike resolution system, which escalates to more expensive/trustworthy systems as the volume or importance of the question goes up)
1Agreed on AI-suggested operationalizations and resolutions (this could maybe be iterated on more extensively on more classical prediction market sites).
I'd be interested in you reading over my reply to Nathan (which maybe does a better job laying out why I care about this), and then maybe spending a couple minutes on rambling about "does the proposal here feel like it'd work? Or, do you have other ideas for solving the high level goal of "move the needle on 'mainstream incentives towards truth'?".
Low hanging fruit: There used to be a bot ("RemindMeOfThis" or similar) people could tag to get a reminder in some time frame. For some reason I don't see a currently working version of this. I assume a contributing factor is that the X API is now expensive.
To be clear, your proposal does not include a prediction market of any sort, just AI automated operationalizations + user voted results?
I think giving twitter users some internet points they can use to bet on the markets, like with manifold, may just work. But it's possible that the public won't use it in a useful way compared to user voting.
Yeah my impression is that prediction markets incentivize things that are more like "seek out poorly priced things and arbitrage them" rather than "make useful, correct predictions that anyone cared about." (I have some vague sense that the shape of prediction markets tends towards not solving the problems I care about, although I haven't thought about it too hard).
There totally could also be prediction markets involved here. But the primary purpose is to establish a record of "who predicted accurate things that anyone cared about" so that can become an easily referenced source-of-authority.
This needs to be incentive-compatible with "being a pundit", I think. Like the idea is to make it so people who do pundrity on twitter are automatically nudged to make their predictions legible and gradable.
This needs to be incentive-compatible with “being a pundit”, I think. Like the idea is to make it so people who do pundrity on twitter are automatically nudged to make their predictions legible and gradable.
The default is of course that people who do punditry are automatically nudged to make their predictions illegible, because the existence of any such system threatens to embarrass the many pundits who know their predictions are bullshit.
This seems like a huge problem and I don't know how you get around it, other than hoping the system takes off so well that people can be shamed for conspicuously dodging its accountability.
I agree it's quite difficult to solve this completely. But, it seems like you could make it so "On twitter, the default thing is if you try to pundit, the system nudges you towards legiblizing your predictions."
This at least makes more of the world, on the margin, track "being correct" as a good thing to do. This can hopefully have flowthrough effects as the equilibrium shifts from "illegible pundits just look like status quo" to "illegible pundits look weirder."
Is there some kind of "get prediction markets, or predictions, onto twitter as a central object" project going? If so, how is it going?
I'm thinking through "how to raise median sanity" on a world scale. There are several incentive and institutional problems that make this very difficult. One angle is "try to make it a thing that the world tracks and cares about your predictions, and getting them right/wrong."
Two past angles here were:
An idea for a twitter feature, which I think Musk should conceptually like given his stated values, and, I think he might have enough inertia to get behind, is:
So, the final result will give a succinct "do The People think this resolved true?", which is some kind of percentage score that takes into account "how much it seems true to people" (if it's ambiguous / on-a-spectrum), and controversial the answer is. (TODO: figure out good mathy operationalization that is robust to partisanship)
But then, immediately underneath that, is a list of citations/evidence, so even if something is ambiguous, it's as easy as possible to see concrete facts related to it. (Also with community-notes-esque voting on relevance)
"Have you successfully made good predictions" is a badge / leaderboard that is optimized to be fun and rewarding.
The "community-notes-for-vague-prediction-resolution" mechanic could be implemented by Manifold or Metaculus or LessWrong or whoever, but, I think it's a lot more useful if it's on twitter. (tho maybe good to test it out on these places to iron kinks out and prototype UI).
My impression of the landscape is "Musk did really care about but truth, and believes in community notes in particular, but also went crazy after buying twitter. The Community Notes team is a small fiefdom he trusts and still believes in." So, insofar as this is pitched as an extension of that, might be doable.
Curious about takes from @Zvi and @Austin Chen.