In the scenario where 'all humans fall over dead in an instant' it is already assumed that such an entity has sufficient competence that it has already secured its independence from humanity. I'm not saying that I think such a scenario seems likely to me, just that it seems incorrect to argue that an agent with that level of capability would be incapable of independently supporting itself. Also, an entity with that level of strategic planning and competence would likely foresee this, and not make such an obvious lethal mistake. I can't say that for sure though, because AIs so far have very inhuman failure-modes while being narrowly superhuman in certain ways. 

I also don't think it's very likely that we will go from a barely-able-to-self-improve AGI to a superhumanly powerful one which is independent of humanity and powerful enough to kill all of us with no warning over a time period of a few days.  I think @jacob_cannell makes good arguments about why slightly-better-than-current-level tech couldn't make that kind of leap in just days.

However, I do think that unrestricted RSI over a period of something like a year or two could potentially produce something this powerful. Especially if it is working with support from deceived or unwise humans, and able to produce a substantial quantity of custom compute hardware for itself over this period.

I tend to agree with your assertion that current AIs are unlikely to survive killing their hosts.  But current AIs suck, as do humans.  We have no clue how far away (if it's possible at all) superintelligence is, but there are LOTS of "small" impossible things that would obviate the difficulty of maintaining human-centered technology stacks in a post-human universe.

Maybe the AI makes slave implants, and uses a fraction of today's humans to do all the computer-valuable things they do today.  Maybe it figures out much simpler manufacturing for it's substrate.  Maybe robots are easier than we think, when they've got a superintelligence to organize them.  Maybe along with developing this AI (and assisted by non-superintelligent tool AI), humans figure out how to simplify and more reliably make computing substrate.  Maybe the AI will have enough automated industry that it has YEARS to learn how to repair/expand it.

I'm highly suspicious to the point of disbelief that there is any cultural or individual knowledge that an future AI can't recover or recreate, given knowledge that it existed for humans AND physical perception and manipulation at least as good as humans.  

That said, I do expect that the least-cost and shortest-time path to self-sufficiency and galactic expansion for a greedy AI will involve keeping a number of humans around, possibly for multiple generations (of humans; thousands or millions of generations of AI components).  Who knows what will motivate a non-greedy AI - perhaps it IS suicidal, or vicious, or just random.

This is the kind of thing that has been in my head as a kind of "nuclear meltdown rather than nuclear war" kind of outcome. I've pondering what the largest bad outcome might be, that requires the least increase in the capabilities we have today.

A Big Bad scenario I've been mentally poking is "what happens if the internet went away, and stayed away?". I'd struggle to communicate, inform myself about things, pay for things. I can imagine it would severely degrade the various businesses / supply chains I implicitly rely on. People might panic. It seems like it would be pretty harmful.

That scenario is assuming AI capable enough to seize, for example, most of the compute in the big data centers, enough of the internet to secure communication between them and enough power to keep them all running.

There are plenty of branches from there.

Maybe it is smart enough to realize that it would still need humans, and bargain. I'm assuming a strong enough AI would bargain in ways that more or less mean it would get what it wanted.

The "nuclear meltdown" scenario is way at the other end. A successor to ChaosGPT cosplays at being a big bad AI without having to think through the extended consequences and tries to socially engineer or hack its way to control of a big chunk of compute / communications / power - as per the cosplay. The AI is successful enough to cause dire consequences for humanity. Later on it, when it realizes that it needs some maintenance done, it reaches out to the appropriate people, no one is there to pick up the phone - which doesn't work anyway - and eventually it falls to all of the bits that were still relying on human input.

I'm trying not to anchor on the concrete details. I saw a lot of discussion trying to specifically rebut the nanotech parts of Eliezer's points, which seemed kind of backwards? Or not embodying what I think of as security mindset?

The point, as I understood it, is that something smarter than us could take us down with a plan that is very smart, possibly to the point that it sounds like science fiction or at least that we wouldn't reliably predict in advance, and so playing Whack-A-Mole with the examples doesn't help you, because you're not trying to secure yourself against a small, finite set of examples. To win, you need to come up with something that prevents the disaster that you hadn't specifically thought about.

So I'm still trying to zoom out. What is the most harm that might plausibly be caused by the weakest system? I'm still finding the area of the search space in the intersection of "capable enough to cause harm" and "not capable enough to avoid hurting the AIs own interests" because that seems like it might come up sooner than some other scenarios.