Richard, thanks for your reply. Just for reference, I think this goes under argument 5, right?It's a powerful argument, but I think it's not watertight. I would counter it as follows:
Minimum hardware leads to maximum security. As a lab or a regulatory body, one can increase safety of AI prototypes by reducing the hardware or amount of data researchers have access to.
My response to counterargument 3 is summarized in this plot, for reference: https://ibb.co/250Qgc9
Basically, this would only be an issue if postponement cannot be done until risks are sufficiently low, and if take-off would be slow without postponement intervention.
Interesting line of thought. I don't know who and how, but I still think we should already think about if it would be a good idea in principle.Can I restate your idea as 'we have a certain amount of convinced manpower, we should use it for the best purpose, which is AI safety'? I like the way of thinking, but I still think we should use some of them for looking into postponement. Arguments:- The vast majority of people is unable to contribute meaningfully to AI safety research. Of course all these people could theoretically do whatever makes most money and then donate to AI safety research. But most will not do that in practice. I think many of these people could be used for the much more generic task of convincing others about AI risks, and also arguing for postponement. As an example, I saw a project once with the goal of teaching children about AI safety which claimed they could not continue for lack of 5000$ of funding. I think there's a vast sea of resource-constrained possibility out there once we make the decision that telling everyone about AI risk is officially a good idea.- Postponement weirdly seems to be a neglected topic within the AI safety community (for dislike of regulation, I guess), but also outside the community (for lack of AI risk insight). I think it's a lot more neglected at this point than technical AI safety, which is perhaps also niche, but does have its own institutes already looking at it. Since it looks important and neglected, I think an hour spent on postponement is probably better spent than an hour on AI safety, perhaps unless you're a talented AI safety researcher.
Thanks for that comment! I didn't know Bill McKibben, but I read up on his 2019 book 'Falter: Has the Human Game Begun to Play Itself Out?' I'll post a review as a post later. I appreciate your description of what the scene was like back in the 90s or so, that's really insightful. Also interesting to read about nanotech, I never knew these concerns were historically so coupled.
But having read McKibben's book, I still can't find others on my side of the debate. McKibben is indeed the first one I know who both recognizes AGI danger, and does not believe in a tech fix, or at least does not consider this a good outcome. However, I would expect that he would cite others on his side of the debate. Instead, in the sections on AGI, he cites people like Bostrom and Omohundro, which are not postponists in any way. Therefore I'm still guessing at this moment that a 'postponement side' of this debate is now absent, and it's just that McKibben happened to know Kurzweil who got him personally concerned about AGI risk. If that's not true and there are more voices out there exploring AGI postponement options, I'd still be happy to hear about it. Also if you could find links to old discussions, I'm interested!
Thanks for your insights Adam. If every AGI researcher is in some sense for halting AGI research, I'd like to get more confirmation on that. What are their arguments? Would they also work for non-AGI researchers?
I can imagine the combination of Daniel's point 1 and 2 stops AGI researchers from speaking out on this. But for non-AGI researchers, why not explore something that looks difficult, but may have existential benefits?
I agree and thanks for bringing some nuance in the debate. I think that would be a useful path to explore.
I'm imagining an international treaty, national laws, and enforcement from police. That's a serious proposal.
I think a small group of thoughtful, committed, citizens can change the world. Indeed, it is the only thing that ever has.
I appreciate the effort you took in writing a detailed response. There's one thing you say in which I'm particularly interested, for personal reasons. You say 'I've been in or near this debate since the 1990s'. That suggests there are many people with my opinion. Who? I would honestly love to know, because frankly it feels lonely. All people I've met, so far without a single exception, are either not afraid of AI existential risk at all, or believe in a tech fix and are against regulation. I don't believe in the tech fix, because as an engineer, I've seen how much of engineering is trial and error (and science even more). People have ideas, try them, it says boom and then they try something else. Until they get there. If we do that with AGI, I think it's sure to go wrong. That's why I think at least some kind of policy intervention is mandatory, not optional. And yes it will be hard. But no argument I've heard so far has convinced me that it's impossible. Or that it's counterproductive.
I think we should first answer the question: is postponement until safety a good idea if it would be implementable. What's your opinion on that one?
Also, I'm serious: who else is on my side of this debate? You would really help me personally to let me talk to them, if they exist.