This is a special post for quick takes by niknoble. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
These imply that once we gain the ability to select our personalities by directly manipulating the brain, there will suddenly be strong consensus on certain political issues that were previously controversial.
I suspect this implication is false. You'd need to enumerate the traits and figure out correlation for each between #1 and #2 to know just how false.
My strong expectation is that the personality traits that are popular aren't REALLY what people would like to have - they're what they'd like others to have, and what they'd like to be seen as having. And that the traits with strong political correlation (unsure of causality, of course) are different than the ones people profess to wanting.
I also suspect "the ability to select our personalities..." is deeply unlikely in any straightforward way. It's ALREADY the case to some extent that intentional cognitive therapy and environment changes can change one's personality traits somewhat, and very few choose to do so. The ability to select OTHER's personalities is much larger (we call it parenthood), and there's no real consensus on what's desirable there either.
Paranoia is not entirely independent of whether they're actually out to get you.
I often wonder if there is a No-Alignment Theorem that says you can't always control the actions of an intelligent entity. Maybe something with the flavor of the undecidability of the halting problem or Godel's Incompleteness Theorems, where the issue stems from the fact that an intelligent entity can model itself and reflect from a distance on the goals you've given it.
I doubt such a thing exists, but it's fun to think about. It would also require a mathematical formulation of an intelligent entity, which seems to be quite a ways off. And even if such a theorem does exist, it would almost certainly be irrelevant for doing alignment in practice, the same way Godel's Incompleteness Theorems do not affect the day-to-day work of mathematicians.
It's now becoming easy to have your computing hardware run any software you want. This makes software less valuable because it's easier to create, and it makes computing hardware more valuable because it can be used more effectively.
Analogue for the physical world:
It will eventually be easy to arrange raw materials into any shape you want. This will make complex physical objects (including computing hardware) less valuable because they'll be easier to create, and it will make raw materials more valuable because they'll be able to be used more effectively.
The synthetic population bomb:
In a world of abundance, it will be easy to create digital people, and those people will have as much of a claim to resources as anyone else. If we create people faster than we extract new resources, then the pie grows but each person's slice shrinks. In the end, we live in a technological wonderland but own fewer and fewer atoms.
it will be easy to create digital people, and those people will have as much of a claim to resources as anyone else
It seems necessary in such a world to require that the necessary resources are provisioned before new people are created, as a precondition of it being OK to create new people.
then the pie grows but each person's slice shrinks
If specifically those who decide to create new people have to provide for them, then only their pieces of the pie shrink. This way the incentives balance out Malthusian redistribution.
Good points. It's still pretty fraught though:
Funding your own existence doesn't lead to Malthusian issues, if it's not at the expense of those who didn't consent to this externality. The reachable universe though won't be getting any more resources, so property rights should be about shares in the reachable universe, to prevent a race to claim everything using new entities, at the expense of those who are not as immediately grabby (which can be a subtle long term alignment issue even if blatant takeover is averted).
Funding your own existence doesn't lead to Malthusian issues, if it's not at the expense of those who didn't consent to this externality.
Had to think about this for a while, but I'm assuming this means that by funding your own existence through consensual economic transactions, you're necessarily generating as much value as you consume, so you're not reducing the amount of pie available for everyone else.
That seems vaguely reasonable. I guess an important thing is then that new people aren't able to extract any resources through purely political means, which goes to your point about having hard limits on who provides for the new people.
Somehow it still feels suspicious. Like, if today we introduce a trillion initially-broke self-funding humans into the world, who all need a place to live, surely that makes it harder for me to rent an apartment, right? I'll admit that I know very little about economics.