This is a special post for quick takes by esotericn. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Thoughts on inherent paradoxes between welfare/redistribution and ASI.
To first order, ASI ends up resulting in everyone being on welfare, e.g. the "best case" ASI that I am aware of is the one in which ASI runs the future and sort of keeps humans around as pets and indulges our whims which seems pretty analogous to me; we just kind of all do whatever and get fed and watered anyway.
My summary of the arguments against significant general state sponsored welfare, by that I mean, within a developed country, UBI or something that exists at a level significantly above helping the severely disabled.
1. serves as part of the reward function for doing useful work. do no work = get no money. we want people to do work, so we tie work to money and survival and say, if you are capable, go out there and do things.
2. requires some sort of redistribution whether that is via taxation or inflation or really just something, it requires moving resource from more productive to less productive somehow.
3. kind of same as 1, we want to reward intelligence in some way and so having outcome (money) strongly tied in both directions to this feels sensible.
_However_ in the ASI future, no-one is useful in the practical perspective. therefore:
1 is irrelevant 2 the work done by the ASI is OOMs higher than the total combined output therefore differentially there is not really a meaningfully "less" or "more" productive segment to redistribute any more
The biggest one that sticks out to me though, and most interesting, is "3".
ASI is the ultimate endgame of more intelligence. There's a paradox here; it's like we want more intelligence, but.. not like that. Hmm.
Possible resolution: we don’t want “more ability / intelligence” as this abstract thing; we want more aligned intelligence, or a kind of “useful person for humanity“ maximiser, e.g we don’t want a system in which money preferentially goes to mad scientists in the form of massive salaries to invent new forms of weapon.