Agreed! I don’t see if there is a quick “hack” that will enable ASI to harness that negentropy though. The "cliff" I had in mind would look like creating a sizable competitive advantage with only small scale control.
E.g. the first bacteria to perform photosynthesis locally changed a single molecule and had an advantage that its competitors could neither match nor negate. I wonder how ASI could pull the rug from beneath us in a comparable manner.
Something that is not often discussed explicitly and factors into the different intuitions people have about P(Doom) is how close to optimal biology and humans are in terms of harnessing negative entropy. This consideration pertains to equally to nanobots, ASI and artifical life in general.
Let's consider grey goo first: The race to turn all resources into copies of yourself has been going for a few billion years and is quite competitive. In order to supplant organic life, nanobots would have to either surpass it in carnot efficiency or (more likely) utilise a source of negative entropy thus far untapped. Examples of this previously happening are:
Of course! The way I think of it, violence would be using other lifeforms as sources of negentropy.
I like the invasive species argument, I agree that we would be very vulnreable to an engineered pathogen.