Jason Fantl's Shortform

Jason Fantl

Jason Fantl's Shortform — LessWrong

Jason Fantl's Shortform

15th Oct 2026

1 min read

2

This is a special post for quick takes by Jason Fantl. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.

Mentioned in

67How I discovered an Exoplanet & the science apocalypse

Jason Fantl's Shortform

13 comments, sorted by

top scoring

Click to highlight new comments since: Today at 7:08 AM

[-]Jason Fantl14d4630

We may soon have a need to de-duplicate safety research. I recently spoke with the CEO of an AI safety nonprofit, and he said that a large part of his work is coordinating with others orgs to ensure he isn't accidentally working on the same project. Oftentimes you don't hear about work until it is finished, so it is very easy for work to be partially duplicated when lots of people are working in a similar area. I expect we will see am amplified version of this problem with autonomous researchers. Imagine a Claude in every lab, all looking at the same public research, and all deciding on the same "most promising" research direction. We will need some coordination mechanism once autonomous AI researchers are picking their own research directions.

[-]_rpd14d2416

replication is important though

[-]Daniel Samuel14d190

If anyone starts working on this, please reply here so the rest of us don’t end up duplicating the de-duplication effort.

[-]leogao13d236

this joke has already been made before (schmidhuber, 1989) and therefore is duplicating the joking about duplicating the deduplication effort effort

[-]rcc14d53

Agree we need a coordination mechanism, but I think the goal should be coordination, not de-duplication.

Multiple groups working on the same topic can be beneficial. We get different approaches, replication and incentives to produce the best research. We also get a chance to figure out what "best" even means. Maybe it's whoever is quickest, or who most comprehensively covers available models, or whose research is the easiest to reproduce - nobody knows that yet.

Coordination would be helpful in that it makes the duplication decisions intentional, not incidental.

[-]Oz Amram14d40

Agreed. A hub that was well organized where people could post projects they are working on with rough descriptions could be valuable here. Also to find collaborators.

And even post open concrete project ideas that would be useful for newcomers to work on would be useful. Expect a lot of researchers coming from other fields (math, physics, CS) in the near term who can do good work but need to be pointed to useful projects

[-]Karan Bansal14d10

I am afraid for that to work we would need to have a system that doesn’t necessarily reward people monetarily or otherwise working on problems with immediate effect. Or else, the sense of competition is very hard to get rid of.

[-]Arjun Panickssery13d20

> I recently spoke with the CEO of an AI safety nonprofit, and he said that a large part of his work is coordinating with others orgs to ensure he isn't accidentally working on the same project.

Is this level of coordination actually socially optimal? The nonprofit sector already has a number of incentive structures that reduce output. Maybe some competition is a good thing even between nonprofits funded by the same donor.

[-]Manu Thaisseril14d20

What do you think is the right approach to solve this? People have tried building an AI safety map, but it did not work out very well for various reasons, because the map gets stale really fast and the map only tracks what is legible and already otherwise visible to the researchers. How do you think a sustainable solution would look?

[-]Brendan McGarry13d10

I think this could be resolved if AI safety researchers maintain two habits:
1. Post at the start of new work about what they're working on.
2. Begin decisions about new work with searching (perhaps via an effective LLM) for potential ongoing work in the considered domains.

That way duplication of work becomes a choice (e.g. replication) rather than an accident.

If this seems sensible then an important detail would be to ensure you're posting on a platform that gets picked up by effective AI search methods.

[-]Jason Fantl17d*91

I want to share a pitch for working in AI safety that seems to be more effective then other pitches I have tried in the past. This is to people who are already AGI-pilled, but for some reason still reluctant to work in AI safety (a surprising number of people I meet are like this).

The pitch goes like this: "You sometimes get existential dread, really sucks right? I've gone through that, still do sometimes, but I found something that helps. If you work on something you think might help move the needle on AI safety, even a little bit, it starts to replace that dread with a small amount of hope. Keep in mind that this can't be performative, this only works if you actually believe what you are doing could help. Even though you know you haven't solved the problem, the dread starts to fade. This has worked really well for me, I feel a lot better in my day-to-day. It doesn't make complete sense, your contributions likely don't change the outcome by much, but it will make you feel way better. And who knows, maybe that small contribution will actually make a difference."

It feels weird to frame it in this selfish manner, but people seem more receptive to it. My previous pitches were along the lines of "The world needs saving, its time for us to all do something" or "You actually can do something, it's not hopeless".

Anyone else have pitches they have found effective when speaking to AGI-pilled individuals?

[-]MichaelDickens16d21

FWIW this doesn't work for me. I feel better when I'm working on things that have nothing to do with AI safety. Actively thinking about AI safety makes me feel bad, even if I think I'm (marginally) making things better. Donating to AI safety orgs also makes me feel bad.

[-]Jason Fantl16d40

I wonder if we could make models that imitate humans (similar to Prover-Verifier Games improve legibility of LLM outputs (Kirchner et al.)) from different backgrounds and levels of intelligence, then use these to gate AI written academic papers to stay human readable. For example, in the same way peer-review is meant to gate for research quality, we might give the human-level models a quiz about a submitted paper to gate for readability. Ideally, the different backgrounds and intelligence levels let different journals set different requirements.

Moderation Log