Summary

Can LLMs science? The answer to this question can tell us important things about timelines to AGI. In this small pilot experiment, we test frontier LLMs on their ability to perform a minimal version of scientific research, where they must discover a hidden rule about lists of integers by iteratively generating and testing hypotheses. Results are ambiguous: they're mostly pretty bad at it but top systems show apparent signs of life. We're working on a larger, more rigorous experiment, and we really want your input.

Structure

In this post we:

Describe an experiment on general reasoning and scientific research ability in LLMs.
Describe the main research project for which this is a pilot project.
Ask for your predictions on the outcome of the main project and what you believe it will say about general reasoning in LLMs.

This is followed by appendices with more detail (eg related work, limitations) but we've kept the main body as short and direct as possible.

Introduction

Over the past six months we have been trying to better understand the degree to which LLMs are capable of general reasoning^[1] (in LLM Generality is a Timeline Crux, and LLMs Look Increasingly Like General Reasoners). In short: researchers, including AI safety researchers, have widely differing positions on whether (and to what degree) LLMs are capable of the same sort of accurate and broadly applicable reasoning ability that humans are. At one extreme is the stochastic parrot hypothesis that "no actual language understanding is taking place in [LLMs]" and any apparent signs of reasoning are actually a kind of cheating. At the other extreme is the view that LLMs are already fully capable of all the same types of reasoning as humans are and just need a bit more scaling.

An important negative consequence of this disagreement is that the AI safety community's limited resources are spread across a range of timelines, in a way that impacts the scaling labs much less. The best use of those resources is significantly different if LLMs will scale straight to AGI^[2], relative to the worlds where substantial further breakthroughs are needed first^[3].

In order to try to improve our understanding of this issue and help resolve the disagreement, we created the Numberwang^[4] experiment.

The experiment

This is a pilot experiment for a larger research project which started this month and runs through...

Posts

Wikitag Contributions

Comments