This post is part of the work done at Conjecture.

Epistemic Status: Palimpsest

Better epistemology should make you stronger.

Which is why at Conjecture's' epistemology team we are so adamant on improving our models of knowledge-production: this feels like the key to improving alignment research across the board, given the epistemological difficulties of the field.

Yet we have failed until now to make our theory of impact legible, both to ourselves and to well-meaning external reviewers. The most sorely missing piece is the link between better models of knowledge-production and quick improvements of alignment research, in the shorter timelines that we expect at Conjecture .

Following interviews that we conducted with a handful of alignment researchers (John Wentworth, Vanessa Kosoy, Evan Hubinger, Abram Demski, Steve Byrnes, Conjecture researchers, and Refine participants), we want to present our current best guess for framing how our epistemology research can make alignment research stronger: revealing, analyzing, and expanding or replacing what we call "Cached Methodologies" — patterns that encode how research is supposed to proceed in a given context, for example the idea that we need to prove a statement to learn if it's true or not. Given that this involves bringing to light and questioning cached thoughts about methodology, we dub this approach methodological therapy.

Note that we definitely also want to leverage better models of knowledge-production to help field-builders and newcomers; our current focus on explicit applications for established researchers comes from two key factors: we're currently only three people in the epistemology team, which means we have to prioritize; and we expect that models and tools useful to established researchers will prove instrumental in building ones for field-builders and newcomers.

On credit: this idea emerged from discussions within the epistemology team (composed of Adam Shimi, Lucas Teixeira, and Daniel Clothiaux); Adam contributed the overall perspective, the unification and the therapy metaphor (from Bachelard); Lucas contributed the focus on actual cached patterns in researchers' minds and interviews; Daniel contributed the expansion of the idea to encompass more alternatives (for examples more ways of doing experiments) as well as many insights from the literature. This post is written by Adam; "we" means that the ...

jorgejungle

jorgejungle

jorgejungle

jorgejungle