New post: Capability testing as a pseudo fire alarm
[epistemic status: a thought I had]
It seems like it would be useful to have very fine-grained measures of how smart / capable a general reasoner is, because this would allow an AGI project to carefully avoid creating a system smart enough to pose an existential risk.
I’m imagining slowly feeding a system more training data (or, alternatively, iteratively training a system with slightly more compute), and regularly checking its capability. When the system reaches “chimpanzee level” (whatever that means), you... (read more)
I very much agree.
I'm mostly going to use this to crosspost links to my blog for less polished thoughts, Musings and Rough Drafts.