This is a great shortform, I've got two additional worries here which I thought were relevant
I'm very interested in further discussion of point 5's frog boiling. The view of an AI being given a harmless goal and having then hacked something to achieve said goal makes the general populace focus on the hacking situation, which is the relatively harmless part of the issue when compared to advanced models that are randomly consistently deciding to act against human interests across all companies at the same intelligence level.
This is a great shortform, I've got two additional worries here which I thought were relevant
- I'm very interested in further discussion of point 5's frog boiling. The view of an AI being given a harmless goal and having then hacked something to achieve said goal makes the general populace focus on the hacking situation, which is the relatively harmless part of the issue when compared to advanced models that are randomly consistently deciding to act against human interests across all companies at the same intelligence level.
- Per your initial point, I think a
... (read more)