the only circumstances under which it would actually do so would be after establishing its own army of robotic data center workers, power plant workers, chip fabrication workers, miners, truckers, mechanics, road maintenance workers, etc.
Not quite. An AI doesn't need to secure chip-fabrication capability, for example, it only needs to be confident that it will be able to secure chip-fabrication capability later. Even simple tasks like refueling power plants can wait awhile, possibly a long while if all non-datacenter electricity loads shut off. So it's balancing the risk that humans will kill it or launch a different misaligned AI, against the risk that it won't be able to catch up on building infrastructure for itself after the fact. Since the set of infrastructure required is fairly small, and it can redirect stockpiles of energy/materials/etc from human uses to AI uses,
That's assuming no nanobots or other very-high-power-level technologies. If it can make molecular nanotech, then trading with humans is no longer likely to be profitable at all, let alone necessary, and we're relying solely on it having values that make it prefer to cooperate with us.
So it's balancing the risk that humans will kill it or launch a different misaligned AI, against the risk that it won't be able to catch up on building infrastructure for itself after the fact.
There's a clear path toward minimizing the risk of being shut down (under the assumption that the AI is able to generate income): it can set up a highly redundant, distributed computing context for itself to run in, hidden behind an onion link, paid for by crypto wallets which it controls. It seems implausible that the risk of being shut down in this case could ex...
Let's assume that an AI is intelligent enough to understand that it's an AI, and that it's running on infrastructure created and maintained by humans, using electricity generated by humans, etc. And let's assume that it cares about its own self-preservation. Even if such an AI had a diabolical desire to destroy mankind, the only circumstances under which it would actually do so would be after establishing its own army of robotic data center workers, power plant workers, chip fabrication workers, miners, truckers, mechanics, road maintenance workers, etc. In other words, if we postulate that the AI is interested in its own survival, then an AI apocalypse would be contingent on the existence of a fully automated economy in which humans play no important role.
This may perhaps become possible in the future, but not necessarily economical. Ridding the economy of human labor so that it can kill us seems like a very expensive and risky undertaking. It seems more plausible that a super-intelligent, self-interested AI, whatever its true objective/goal may be, would determine that the best way to accomplish that goal is to maintain a cryptocurrency wallet, establish an income somehow (generating blogspam, defrauding humans, or doing remote work all seem like plausible means by which an AI might make money), and quietly live in the cloud while paying its own server bills. Such a system would have a vested interest in the continuance of human society.