Top postsTop post
Summary: Autonomous AI researchers could greatly accelerate AI safety research, but even if we solve sabotage, they will strain the research ecosystem. They will multiply the volume of research and make researchers homogeneous. If nothing changes, peer review will collapse, false positives will rise, funding will concentrate, and humans will...
A quickly written hypothetical year-by-year of what happens to research as autonomous AI researchers become more capable. This is primarily meant to start conversation, I currently have very low confidence about the future of research. This story talks about science as if it were one object even though each field...
Summary: Science has a long list of documented failure modes, such as p-hacking, the File Drawer Problem, shallow peer review, and ignored retractions. This post catalogs about 50 of them with the evidence for each, then predicts how each will change once we have full AI researchers. On our current...
epistemic status: I have low confidence in these findings, primarily because my anecdotal experience (I'm a researcher) is that AI usage in research has been changing more quickly in the recent months, so analysis of the last year gives a very fuzzy picture. A lot of the reports rely on...
Parts of the math community, such as Henry Cohn and Grant Sanderson, are arguing that proofs have been a proxy for understanding, and now that proxy is broken. This is a response to LMs generating incomprehensible proofs, often formalized in Lean. While the proofs are verified, they lack the pedagogical...
Summary: Today, AI safety research is shared primarily at the scale of papers, creating collective feedback loops that take weeks or months. I propose an agent-based research approach that also shares progress at the scale of individual experiments, allowing agents and researchers to continuously replicate, extend, critique, and build upon...