That may sound like a threat, it is. But not from me. Here is the problem, on a long enough timeline an "unaligned" AI is bound to happen. That happens whether we get to ASI before we solve alignment or not. Here are a couple of scenarios:
- Some wunderkind in their basement/company solves ASI before anyone else and before anyone else even knows it's too late.
- We successfully solve alignment and ASI. We aligned it to our current ideals. But it is what I think of as "Brittle ASI". Where the whole thing is a cluster of it just works and changes lead to out right errors or just different.
- Alignment and... (read 493 more words →)
This can be addressed by peer to peer tech and federation. Peertube uses few techniques, that make it more tentable: hosting on the site itself, site to site sharing (activity pub), and bittorent to fill the massive demand. The bittorent part is on by default and there are already more than then an few instances which people can share on.
Yes. That is one of the things in possibility space. I don't think unaligned means safe. We work with unaligned people all the time, and some of them aren't safe either.
The main thing I was hoping people would understand from this is that an unaligned AI is near a 100% possibility. Alignment isn't a one and done goal that so many people act like it is. Even if you successfully align an AI, all it takes is one failure to align and the genie is out of the bottle. One single point of failure and it becomes a cascading failure.
- So let's imagine an ASI that works on improving itself. How does it
... (read more)