Grant Demaree

This piece, which predates ChatGPT, is no longer endorsed by its author. Eliezer's recent discussion on AGI alignment is not optimistic. > I consider the present gameboard to look incredibly grim... We can hope there's a miracle that violates some aspect of my background model, and we can try to prepare for that unknown miracle For this post, instead of debating Eliezer's model, I want to pretend it's true. Let's imagine we've all seen satisfactory evidence for the following: 1. AGI is likely to be developed soon* 2. Alignment is a Hard Problem. Current research is nowhere close to solving it, and this is unlikely to change by the time AGI is developed 3. Therefore, when AGI is first developed, it will only be possible to build misaligned AGI. We are heading for catastrophe How we might respond I don't think this is an unsolvable problem. In this scenario, there are two ways to avoid catastrophe: massively increase the pace of alignment research, and delay the deployment of AGI. Massively increase the pace of alignment research via 20x more money I wouldn't rely solely on this option. Lots of brilliant and well-funded people are already trying really hard! But I bet we can make up some time here. Let me pull some numbers out of my arse: * $100M per year is spent per year on alignment research worldwide (this is a guess, I don't know the actual number) * Our rate of research progress is proportional to the square root of our spending. That is, to double progress, you need to spend 4x as much** Suppose we spent $2B a year. This would let us accomplish in 5 years what would otherwise have taken 22 years. $2B a year isn't realistic today, but it's realistic in this scenario, where we've seen persuasive evidence Eliezer's model is true. If AI safety is the critical path for humanity's survival, I bet a skilled fundraiser can make it happen Of course, skillfully administering the funds is its own issue... Slow down AGI development The problem, as I und