Tyler Cowen earlier this year wrote about the intellectual demise of libertarianism (Cowen 2020). I'm here wondering: aside from its normative status, is libertarianism unsustainable for reasons intrinsic to itself? Said differently, is it true that there are mechanisms intrinsic to libertarianism that forestall its ability to exist? To stimulate...
Whereas companies often undergo a friendly merger or acquisition, why don't countries more often do this? Set aside colonialism because I'd like to focus on the case where the threat of violence is not paramount. I suppose the right incentives aren't in place but is this an intrinsic fact of...
Question. In the world in which multiple AGIs obtain (i.e., not a singleton), is it likely these AGIs will be parents/mentors/guardians of younger AGIs? Context. I was wondering whether parenthood (or, more minimally, guardianship) is an intrinsic property of the evolution of sufficiently intelligent agents. The answer seems to depend,...
Aha. Ugh, what an unfortunate sequence of events.
Any explanations for why Nick Bostrom has been absent, arguably notably, in recent public alignment conversations (particularly since chatgpt)?
He's not on this list (yet other FHI members, like Toby Ord, are). He wasn't on the FLI open letter, too, but I could understand why he might've avoided endorsing that letter given its much wider scope.
Most of the argument can be boiled down to a simple syllogism: the superior intelligence is always in control; as soon as AI is more intelligent than we are, we are no longer in control.
Seems right to me. And it's a helpful distillation.
... (read more)When we think about Western empires or alien invasions, what makes one side superior is not raw intelligence, but the results of that intelligence compounded over time, in the form of science, technology, infrastructure, and wealth. Similarly, an unaided human is no match for most animals. AI, no matter how intelligent, will not start out with a compounding advantage.
Similarly, will we really have no ability to learn from mistakes? One
Thanks Duncan!
There's a somewhat obscure but fairly-compelling-to-me model of psychology which states that people are only happy/okay to the extent that they have some sort of plan, and also expect that plan to succeed.
What's the name of this model; or, can you point to the fuller version of it? Seems right and would see it fleshed out.
hi Matt! on the coordination crux, you say
The first AGIs we construct will be born into a culture already capable of coordinating, and sharing knowledge, making the potential power difference between AGI and humans relatively much smaller than between humans and other animals, at least at first.
but wouldn’t an AGI be able to coordinate and do knowledge sharing with humans because
a) it can impersonate being a human online and communicate with them via text and speech and
b) it‘ll realize such coordination is vital to accomplish it‘s goals and so it’ll do the necessary acculturation?
Watching all the episodes of Friends or reading all the social media posts by the biggest influencers, as examples.
Tyler Cowen earlier this year wrote about the intellectual demise of libertarianism (Cowen 2020). I'm here wondering: aside from its normative status, is libertarianism unsustainable for reasons intrinsic to itself? Said differently, is it true that there are mechanisms intrinsic to libertarianism that forestall its ability to exist?
To stimulate discussion, the following two hypotheses came to mind that support this notion. Both of these hypothesis posit that actors within the polity will vote for politicians that increase state capacity and thus prevent/terminate libertarianism.
Elite Support Hypothesis. Economic growth in modern societies demands certain investments and solutions to collective action problems that only a state can provide. Wealthy elites will support politicians that increase state capacity to provide these services and these politicians will gain power.
Non-Elite Support Hypothesis. Economic inequality will rise to a point where non-elites will support politicians favoring a welfare state and these politicians will gain power.
You can get many of the benefits of having one country through mechanisms like free trade agreements, open borders, shared currency zones etc.
This is key in my opinion.
Whereas companies often undergo a friendly merger or acquisition, why don't countries more often do this? Set aside colonialism because I'd like to focus on the case where the threat of violence is not paramount.
I suppose the right incentives aren't in place but is this an intrinsic fact of international relations or democratic politics?
For example, a merger between the US and Mexico seems, naively, like it could be in both nations' interest. The US gets more land and lower-cost labor and Mexico gets access to better governance and knowledge spillover.
Duplicates - digital copies as opposed to genetic clones - might not require new training (unless a whole/partial restart/retraining was being done).
Wouldn't new training be strongly adaptive -- if not strictly required -- if the duplicate's environment is substantively different from the environment of its parent?
When combined with self-modification, there could be 'evolution' without 'deaths' of 'individuals' - just continual ship of Theseus processes. (Perhaps stuff like merging as well, which is more complicated.)
I understand this model; at the same time, however, it's my impression that it's commonplace in software development to periodically altogether jettison an old legacy software system in favor of building a new system from the ground-up. This seemed to be evidence that there are growing costs to continual self-modification in software systems that might limit this strategy.
I'll check it out -- thanks Zachary!
Question. In the world in which multiple AGIs obtain (i.e., not a singleton), is it likely these AGIs will be parents/mentors/guardians of younger AGIs?
Context. I was wondering whether parenthood (or, more minimally, guardianship) is an intrinsic property of the evolution of sufficiently intelligent agents.
The answer seems to depend, at least partly, on the stability of the environment and the returns to learning. My off-the-cuff reasoning follows.
1. It seems that a species (or, perhaps more generically, sufficiently similar information processing systems), can persist over time either by increasing their own lifespan or by creating new instances of the species.
2.a. It seems that if the environment is highly stable, then increasing lifespan... (read 162 more words →)
Isn't Zvi's post an attempt to explain those observations?