Yeah it doesn't seem scary at all, turning off some core routers at the major telecoms will end it.

The example must violate some assumptions behind the theory, but I'm not sure what.

Possibly because there is a harder limit on humans than on AI? Humans don't replicate very well. 

On a second thought, I don't think comparative holds if demand is exhausted. Comparative Advantage(at least the Ricardo version i know of) only focuses on the maximum amount of goods, not if they're actually needed. If there was more demand for paperclips/staples than there is Production by AI(s), Humans would focus on staples and AI (more) on paperclips. 

The example must violate some assumptions behind the theory, but I'm not sure what.

The theory is typically explained using situations where people produce the things they consume. Like, the "human" would literally eat either that 1 paperclip or those 2 staples and survive... and in the future, he could trade the 2 staples for a paperclip and a half, and enjoy the glorious wealth of paperclip-topia.

Also, in the textbook situations the raw materials cannot be traded or taken away. Humans live on one planet, AIs live on another planet, and they only exchange spaceships full of paperclips and staples.

Thus, the theory would apply if each individual human could survive without the trade (e.g. growing food in their garden) and only participate in the trade voluntarily. But the current situation is such that most people cannot survive in their gardens only; many of them don't even have gardens. The resources they actually own are their bodies and their labor, plus some savings, and when their labor becomes uncompetitive and the savings are spent on keeping the body alive...

Consider the competitive advantages of horses. Not sufficient to keep their population alive at the historical numbers.

You are correct.  Free trade in general produces winners/losers and while on average people become better off there is no guarantee that individuals will become richer absent some form of redistribution.

In practice humans have the ability to learn new skills/shift jobs so we mostly ignore the redistribution part, but in an absolute worst case there should be some kind of UBI to accommodate the losers of competition with AGI (perhaps paid out of the "future commons" tax).

If I include the market price of the house I currently live in (minus the remaining mortgage), I am about 1/4 there. I am saying it because people often only think about how much money they have in the bank.

There is a little voice inside me screaming that it is unfair -- that the house is simply a place I am living in (with my family), just the cost of everyday functioning, and the "real wealth" is only what you have above that: the money you could freely spend without ruining your life.

But that fact is that I do own the house, thus I am in a very realistic sense richer than people who don't (and thus have to spend money every month paying rent), and that once my kids grow up, I could actually sell the house and buy something at half of the price, thus converting its price into actual money that I could actually spend (while still having a roof above my head).

Then again, I am probably older (in my 40s) than the average LW reader, I think. It took a few decades to accumulate even that much wealth.

Sorry, that was the wrong link.  I was more thinking of the $34k/year income required to be in the top 1%.  

But $870k is less than the price of a house in SF.

I too read Accelerando.

But I don't think this future is terribly likely.  It's either human annihilation or massive cosmic endowment of wealth.  The idea that we somehow end up on the knife-edge of survival our resources slowly dwindling requires that r* is too finely tuned to exactly 0.