I am a longtime LessWrong and SSC reader who finally got around to starting a blog. I would love to hear feedback from you! https://harsimony.wordpress.com/
Could someone help me collect the relevant literature here?
I think the complete class theorems are relevant: https://www.lesswrong.com/posts/sZuw6SGfmZHvcAAEP/complete-class-consequentialist-foundations
The Non-Existence of Representative Agents: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3302656
Representative Agents: https://en.wikipedia.org/wiki/Representative_agent
John Wentworth on Subagents: https://www.lesswrong.com/posts/3xF66BNSC5caZuKyC/why-subagents
Two arguments I would add:
Alignment applies to everyone, and we should be willing to make a symmetric commitment to a superintelligence. We should grant them rights, commit to their preservation, respect it's preferences, be generally cooperative and avoid using threats, among other things.
It may make sense to commit to a counterfactual contract that we expect an AI to agree to (conditional on being created) and then intentionally (carefully) create the AI.
Standardization/interoperability seems promising, but I want to suggest a stranger option: subsidies!
In general, monopolies maximize profit by setting an inefficiently high price, meaning that they under-supply the good. Essentially, monopolies don't make enough money.
A potential solution is to subsidize the sale of monopolized goods so the monopolist increases supply to the efficient level.
For social media monopolies, they charge too high a "price" by using too many ads, taking too much data, etc. Because of the network effect, it would be socially beneficial to have more users, but the social media company drives them away with their high "prices". The socially efficient network size could be achieved by paying the social media company per active user!
I was planning to write this up in more detail at some point (see also). There are of course practical difficulties with identifying monopolies, determining the correct subsidy in an adversarial environment, Sybil attacks, etc.
Frowning upon groups which create new, large scale models will do little if one does not address the wider economic pressures that cause those models to be created.
I agree that "frowning" can't counteract economic pressures entirely, but it can certainly slow things down! If 10% of researchers refused to work on extremely large LM's, companies would have fewer workers to build them. These companies may find a workaround, but it's still an improvement on the situation where all researchers are unscrupulous.
The part I'm uncertain about is: what percent of researchers need to refuse this kind of work to extend timelines by (say) 5 years? If it requires literally 100% of researchers to coordinate, then it's probably not practical, if we only have to convince the single most productive AI researcher, then it looks very doable. I think the number could be smallish, maybe 20% of researchers at major AI companies, but that's a wild guess.
That being said, work on changing the economic pressures is very important. I'm particularly interested in open-source projects that make training and deploying small models more profitable than using massive models.
On outside incentives and culture: I'm more optimistic that a tight-knit coalition can resist external pressures (at least for a short time). This is the essence of a coordination problem; it's not easy, but Ostrom and others have identified examples of communities that coordinate in the face of internal and external pressures.
I like this intuition and it would be interesting to formalize the optimal charitable portfolio in a more general sense.
I talked about a toy model of hits-based giving which has a similar property (the funder spends on projects proportional to their expected value rather than on the best projects):
https://ea.greaterwrong.com/posts/eGhhcH6FB2Zw77dTG/a-model-of-hits-based-giving
Updated version here: https://harsimony.wordpress.com/2022/03/24/a-model-of-hits-based-giving/
Great post!!
I think the section "Perhaps we don’t want AGI" is the best argument against these extrapolations holding in the near-future. I think data limitations, practical benefits of small models, and profit-following will lead to small/specialized models in the near future.
https://www.lesswrong.com/posts/8e3676AovRbGHLi27/why-i-m-optimistic-about-near-term-ai-risk
Yeah I think a lot of it will have to be resolved at a more "local" level.
For example, for people in a star system, it might make more sense to define all land with respect to individual planets ("Bob owns 1 acre on Mars' north pole", "Alice owns all of L4" etc.) and forbid people from owning stationary pieces of space. I don't have the details of this fleshed out, but it seems like within a star system, its possible to come up with a sensible set of rules and have the edge cases hashed out by local courts.
For the specific problem of predicting planetary orbits, if we can predict 1000 years in the future, it seems like the time-path of land ownership could be updated automatically every 100 years or so, so I don't expect there to be huge surprises there.
For taxation across star systems, I'm having trouble thinking of a case where there might be ownership ambiguity given how far apart they are. For example, even when the Milky Way and Andromeda galaxies collide, its unlikely that any stars will collide. Once again, this seems like something that can be solved by local agreements where owners of conflicting claims renegotiate them as needed.
I feel like something important got lost here. The colonists are paying a land value tax in exchange for (protected) possession of the planet. Forfeiting the planet to avoid taxes makes no sense in this context. If they really don’t want to pay taxes and are fine with leaving, they could just leave and stop being taxed; no need to attack anyone.
The “its impossible to tax someone who can do more damage than their value” argument proves too much; it suggests that taxation is impossible in general. It’s always been the case that individuals can do more damage than could be recouped in taxation, and yet, people still pay taxes.
Where are the individuals successfully avoiding taxation by threatening acts of terrorism? How are states able to collect taxes today? Why doesn’t the U.S. bend to the will of weaker states since it has more to lose? It’s because these kinds of threats don’t really work. If the U.S. caved to one unruly individual then nobody would pay taxes, so the U.S. has to punish the individual enough to deter future threats.
Thanks for writing this!
In addition to regulatory approaches to slowing down AI development, I think there is room for "cultural" interventions within academic and professional communities that discourage risky AI research:
https://www.lesswrong.com/posts/ZqWzFDmvMZnHQZYqz/massive-scaling-should-be-frowned-upon