Top postsTop post
We propose synthetic scalable oversight, a technique for studying scalable oversight by creating graphical abstractions of real-world problems and training tiny models inside these synthetic environments as a proxy for training LLMs at scale. We thank Oliver Richardson, Christian Szegedy, Michael Douglas, and countless others for the many conversations that...
I would like to thank Professor Vincent Conitzer, Caspar Oesterheld, Bernardo Subercaseaux, Matan Shtepel, and Robert Trosten for many excellent conversations and insights. All mistakes are my own. I think that there's a fundamental connection between AI Safety via Debate and Guaranteed Safe AI via topology. After thinking about AI...
Historically, I've gotten the impression from the AI Safety community that theoretical AI alignment work is promising in long timeline situations but less relevant in short timeline ones. As timelines have shrunk and capabilities have accelerated, I feel like I've seen theoretical alignment work appear less frequently and with less...
So I've been thinking about how to assign probabilities to true/false assignments over claims in the context of a probabilistic argument mapping program.[1] Inevitably I've been confronted with the liar's paradox and a million related headaches. I have some tentative ideas on how I'd address these: basically allowing sentences in...
(Inspired by https://stats.stackexchange.com/questions/175153/approximating-pa-b-c-using-pa-b-pa-c-pb-c-and-pa-pb-pc) Imagine you're working at an electronics manufacturing company. There's a type of component your company needs for an upcoming production process. Your company can either produce these components in-house or it can source them from a vendor. Your job is to determine whether or not to produce...
(I thought about this idea on my own before Googling to see if anyone had already written it up. I found something very similar at https://projecteuclid.org/journals/notre-dame-journal-of-formal-logic/volume-50/issue-2/Justification-by-an-Infinity-of-Conditional-Probabilities/10.1215/00294527-2009-005.full so all credit for this line of thinking should go to the authors of that paper. Still, I think that this concept deserves a...