I'm currently working on a research project for MIRI, and I would welcome feedback on my research as I proceed. In this post, I describe the project.
As a part of an effort to steel-man objections to MIRI's mission, MIRI Executive Director Luke Muehlhauser has asked me to develop the following objection:
"Even if AI is somewhat likely to arrive during the latter half of this century, how on earth can we know what to do about it now, so far in advance?"
In Luke's initial email to me, he wrote:
I think there are plausibly many weak arguments and historical examples suggesting that P: "it's very hard to nudge specific distant events in a positive direction through highly targeted actions or policies undertaken today." Targeted actions might have no lasting effect, or they might completely miss their mark, or they might backfire.
If P is true, this would weigh against the view that a highly targeted intervention today (e.g. Yudkowsky's Friendly AI math research) is likely to positively affect the future creation of AI, and might instead weigh in favor of the view that all we can do about AGI from this distance is to engage in broad interventions likely to improve our odds of wisely handling future crises in general — e.g. improving decision-making institutions, spreading rationality, etc.
I'm interested in abstract arguments for P, but I'm even more interested in historical data. What can we learn from seemingly analogous cases, and are those cases analogous in the relevant ways? What sorts of counterfactual history can we do to clarify our picture?
Luke and I brainstormed a list of potential historical examples of people predicting the future 10+ years out, and using the predictions to inform their actions. We came up with the following potential examples, which I've listed in chronological order by approximate year:
1972: The circle of ideas and actions around the The Limits to Growth, a book about the consequences of unchecked population growth and economic growth.
Constantinople managed to keep the secret of Greek Fire.
Roman emperor Constantine the Great picked Constantinople as his new capital because it could be easily defended against barbarians. He did this although (I think) he faced no barbarian threat. This worked out very well.
I think (am not sure) that these examples are too old to have significant relevance to the question of planning for AI. Do you see a connection?
Although Japan had been a very closed society, when the Japanese saw in 1853 how much more advanced the American navy was from its own the Japanese quickly modernized and became one of the few Asian countries to escape western colonization (until after WWII at least).
Maybe I should clarify that we're looking for examples of people pushing for more specific outcomes (preventing climate change, preventing overpopulation, etc.) than improving the economy in general.
How long ago something happened shouldn't be relevant if you are looking to see if our species is capable of implementing certain types of long-term plans.
Making Constantinople the capital of the Eastern Empire and building its defenses represents perhaps the most successful example in all of human history of someone nudging "specific distant events in a positive direction through highly targeted actions or policies."
I was under the impression that EY wants to keep some of what he discovered a secret. Greek Fire represents an historical example of successfully keeping a tech secret, despite that secret having enormous military value.
I'm currently working on a research project for MIRI, and I would welcome feedback on my research as I proceed. In this post, I describe the project.
As a part of an effort to steel-man objections to MIRI's mission, MIRI Executive Director Luke Muehlhauser has asked me to develop the following objection:
In Luke's initial email to me, he wrote:
Luke and I brainstormed a list of potential historical examples of people predicting the future 10+ years out, and using the predictions to inform their actions. We came up with the following potential examples, which I've listed in chronological order by approximate year: