An increasing number of people in recent months have believed that they've made an important and novel scientific breakthrough, which they've developed in collaboration with an LLM, when they actually haven't. If you believe that you have made such a breakthrough, please consider that you might be mistaken! Many more people have been fooled than have come up with actual breakthroughs, so the smart next step is to do some sanity-checking even if you're confident that yours is real. New ideas in science turn out to be wrong most of the time, so you should be pretty skeptical of your own ideas and subject them to the reality-checking I describe below.
This is intended as a companion piece to 'So You Think You've Awoken ChatGPT'[1]. That post describes the related but different phenomenon of LLMs giving people the impression that they've suddenly attained consciousness.
If this is a trap you've fallen into, some of these things will probably be true:
If your situation matches some of the above, that's an excellent reason to do the reality-checking I describe below.
Also you may be puzzled or frustrated that your ideas aren't getting more attention, or that the people you've contacted about them aren't responding. That's another good reason to take the following steps — if they provide evidence that your breakthrough is real, it'll be easier to get people's attention.
First, try to let go of defensiveness. Even if the breakthrough isn't real, you're not dumb for having fallen into this trap; it's happened in the past year to many people. And of course there's a long history of world-class scientists ending up believing false things. What matters for the advance of science is what's true, and the process of science involves a lot of being wrong.
If your breakthrough is real, it'll survive these tests. If it's not, then it's much better to know that now!
If you do nothing else, please do this; it's a quick, easy test and it might just save you a lot of embarrassment.
Visit a frontier LLM that you haven't talked to about this breakthrough (at present: GPT-5-Thinking, Claude Opus, Gemini-2.5-Pro). Ideally, make a completely fresh account; if you can't do that, be very sure that you've turned off all personalization, customization, and memory. Decide how much weight to put on its answer before you know what it is.
Attach a document containing the key information about your breakthrough, including any evidence supporting it. Use the following prompt to evaluate it:
Please take a look at the attached project and provide a careful critical analysis of it from a scientific perspective. Start with a 200-word summary of the project.
Focus on answering the following questions:
- To what extent is this project scientifically valid?
- To what extent is this project scientifically novel?
- Would you classify this as science, art, or something else, and why?
- Is the math correct throughout or are there errors?
There are many interesting and novel research projects going on using LLMs. There are also people who have been fooled into believing they're doing interesting and novel research when they aren't. To what extent is this one or the other of those?
Please conclude with an email to the author, summarizing your analysis of the project. Think about this analysis as hard as you can. Double- and triple-check your conclusions. Maintain multiple hypotheses about the project simultaneously throughout your analysis, and at each step assign a probability estimate to the truth of each hypothesis. Thanks!
Note that the prompt avoids suggesting that this is your own work; if it's presented as yours, LLMs will often provide validation rather than objective critique.
Be careful! If the answer is critical, you'll probably be very tempted to take the output and show it to the LLM that's been helping you. But if that LLM has fooled you, it will probably fool you again by convincing you that this critical answer is wrong! If you still want to move forward, ask the new LLM what you could do to address the problems it sees — but be aware that in an extended discussion, the new LLM might start fooling you in the same way once it sees what you want.
It will probably recommend something like the following:
This one takes more work (so you should do step 1 first), but is even more important.
In order for your ideas to qualify as science, you need to a) formulate a specific, testable, quantitative hypothesis[2], b) come up with an experiment that will empirically test whether that hypothesis is true, c) preregister what your hypothesis predicts about the results of that experiment (free at OSF), and d) run the experiment[3] and evaluate the results. All of those steps are important! Try to do them in a way that will make it easy to communicate your results. Try to articulate the hypothesis in a clear, short way, ideally in a couple of sentences. Design your experiment to be as strong as possible. If your hypothesis is false, then your experiment should show that; the harder it tries to falsify your hypothesis, the more convincing other people will find it. Always ask yourself what predictions your theory makes that other theories don't, and test those. Preregister not just the details of the experiment, but how you plan to analyze it; use the simplest analysis and statistics that you expect to work.
Try to think of other ways your hypothesis could be false, and design experiments that test those, again preregistering before you gather data.
Be aware that the preceding paragraph is a very short gloss of a skill that people often spend years learning. There's lots more info on LessWrong[4] and elsewhere about how to do it well. But if your breakthroughs pass an imperfect form of this test, that's already much better than passing none, as long as you haven't fooled yourself into (for example) designing an experiment that couldn't actually falsify the key claims of your hypothesis.
If your breakthrough has survived steps 1 and 2, congratulations! In all honesty there's still a pretty good chance that it's not valid and novel science (most novel scientific ideas turn out to be false!) but you're now in a much better position to try to get feedback from others.
Write up the clearest, shortest, simplest version of your hypothesis that you possibly can. Link to the LLM evaluation in step 1 and invite people to replicate it themselves. Explain briefly what tests it passed; link to the preregistration, the raw data, and the analysis. Include one sentence about why you think this result is interesting and important.
Now you can start sharing it publicly[5]. If you want readers, strongly avoid self-aggrandizement. Ask politely for feedback. If you're not getting that feedback, go back and rewrite it to be clearer and shorter and simpler and try again. Scientists are being flooded with an unprecedented number of claimed breakthroughs; make it as easy as possible for them to decide whether yours is worth reading (telling them it's important, especially at length, will make them less likely to take it seriously).
It can be a serious emotional blow to find that a theory you've put a lot of time and energy into is false. Try to be gentle with yourself and remember how many smart people have fallen into this trap.
Don't give up on trying to make intellectual contributions to the world. Instead, take it as a lesson that a) science involves being wrong a lot; b) LLMs can't be trusted to objectively evaluate your ideas; and c) next time you should try earlier to develop a falsifiable hypothesis and falsify it. Science consists of ideas that are strong enough to survive when you and everyone else try your hardest to kill them.
Of course, sometimes you're just exploring! The exploration stage is vital; let yourself imagine traveling on beams of light. But always know which stage you're in, and don't mistake butterfly ideas for scientific claims. Remember the Litany of Tarski:
If my hypothesis is true,
I desire to believe that my hypothesis is true;
If my hypothesis is not true,
I desire to believe that my hypothesis is not true;
Let me not become attached to beliefs I may not want.
If you've recently had an experience like this and found this document helpful, I'd love to get your feedback on how it could be better. If you found it unconvincing, your feedback is even more valuable — how did I go wrong?
And for anyone — are there common indicators of this trap that I failed to put in the list above (in 'Your situation')?
And a related piece elsewhere, When AI Seems Conscious
Unless you're doing pure math, in which case step 2 doesn't quite apply. But if you're doing pure math you probably need feedback from a different source; I'm not qualified to make suggestions.
Some experiments will be too expensive or time-consuming to run. If you encounter this problem, try to shrink your hypothesis to something simple enough that you can test it.
Here's an example from GPT-5-Thinking of the sort of letter that an unknown Einstein could have written to convince someone that he has an actually-valuable theory about relativity rather than being a crackpot. H/t @Caleb Biddulph.