geoffreymiller

The heritability of human values: A behavior genetic critique of Shard Theory

Overview (TL;DR): Shard Theory is a new approach to understanding the formation of human values, which aims to help solve the problem of how to align advanced AI systems with human values (the ‘AI alignment problem’). Shard Theory has provoked a lot of interest and discussion on LessWrong, AI Alignment Forum, and EA Forum in recent months. However, Shard Theory incorporates a relatively Blank Slate view about the origins of human values that is empirically inconsistent with many studies in behavior genetics indicating that many human values show heritable genetic variation across individuals. I’ll focus in this essay on the empirical claims of Shard Theory, the behavior genetic evidence that challenges those claims, and the implications for developing more accurate models of human values for AI alignment. Introduction: Shard Theory as an falsifiable theory of human values The goal of the ‘AI alignment’ field is to help future Artificial Intelligence systems become better aligned with human values. Thus, to achieve AI alignment, we might need a good theory of human values. A new approach called “Shard Theory” aims to develop such a theory of how humans develop values. My goal in this essay is to assess whether Shard Theory offers an empirically accurate model of human value formation, given what we know from behavior genetics about the heritability of human values. The stakes here are high. If Shard Theory becomes influential in guiding further alignment research, but if its model of human values is not accurate, then Shard Theory may not help improve AI safety. These kinds of empirical problems are not limited to Shard Theory. Many proposals that I’ve seen for AI ‘alignment with human values’ seem to ignore most of the research on human values in the behavioral and social sciences. I’ve tried to challenge this empirical neglect of value research in four previous essays for EA Forum, on the heterogeneity of value types in humans individuals, the diversity of v

82Oct 20, 2022

geoffreymiller

Message

Psychology professor at University of New Mexico. BA Columbia, PhD Stanford. Works on evolutionary psychology, Effective Altruism, AI alignment, X risk. Worked on neural networks, genetic algorithms, evolutionary robotics, & autonomous agents back in the 90s.

775

105

Best short introductions to AI safety & alignment for bright college students?

Hi, I've been asked to recommend a couple of short introductions/overviews about the key issues in AI safety and AI alignment. This is will be for the 'Philosophy, Politics, & Economics (PPE) major at Oxford University - which trains some of the brightest undergrads in Britain, many of which go...

Feb 277

My talk on AI risks at the National Conservatism conference last week

Lately I’ve been trying to raise awareness of AI risks among American conservatives. Stopping the reckless development of advanced AI agents (including Artificial General Intelligence (AGI) and Artificial Superintelligence (ASI)) should be a human issue, not a partisan issue. Yet most people working in AI safety advocacy lean to the...

Sep 11, 202552

Biomimetic alignment: Alignment between animal genes and animal brains as a model for alignment between humans and AI systems

(Note: this was also published on EA Forum, May 26, 2023, here; this is a somewhat rough-and-ready essay, which I'll expand and deepen later; I'm welcome any comments, reactions, and constructive criticism.) Overview: To clarify how we might align AI systems with humans, it might be helpful to consider how...

Jun 8, 202310

A moral backlash against AI will probably slow down AGI development

Note 1 : This was posted to EA Forum on May 31 here, as a submission for the 2023 Open Philanthropy AI Worldviews contest, due May 31, 2023. It addresses Question 1: “What is the probability that AGI is developed by January 1, 2043?” Note 2: I'm developing a series...

Jun 7, 202351

The heritability of human values: A behavior genetic critique of Shard Theory

Oct 20, 202282

Brain-over-body biases, and the embodied value problem in AI alignment

Note: This essay was published here on EA Forum on Sept 21, 2022. The description of the brain-over-body biases in the EA subculture may or may not apply to the Rationalist subculture in LessWrong. This essay builds upon this essay on the heterogeneity of human value types. Overview Most AI...

Sep 24, 202210

The heterogeneity of human value types: Implications for AI alignment

Note: This essay was originally posted on EA Forum here on Sept 16. I’d welcome comments from LessWrong readers and AI Alignment Forum experts. I have posted some related essays on EA Forum about the importance for AI alignment of considering corporal/body values, religious values, and the diversity of values...

Sep 23, 202210

Load More (7/8)

LESSWRONG
LW

LESSWRONG
LW

geoffreymiller

geoffreymiller

geoffreymiller

The heritability of human values: A behavior genetic critique of Shard Theory

My talk on AI risks at the National Conservatism conference last week

A moral backlash against AI will probably slow down AGI development

AI alignment with humans... but with which humans?

geoffreymiller

Best short introductions to AI safety & alignment for bright college students?

My talk on AI risks at the National Conservatism conference last week

Biomimetic alignment: Alignment between animal genes and animal brains as a model for alignment between humans and AI systems

A moral backlash against AI will probably slow down AGI development

The heritability of human values: A behavior genetic critique of Shard Theory

Brain-over-body biases, and the embodied value problem in AI alignment

The heterogeneity of human value types: Implications for AI alignment

The heritability of human values: A behavior genetic critique of Shard Theory

My talk on AI risks at the National Conservatism conference last week

A moral backlash against AI will probably slow down AGI development

AI alignment with humans... but with which humans?

Best short introductions to AI safety & alignment for bright college students?

My talk on AI risks at the National Conservatism conference last week

Biomimetic alignment: Alignment between animal genes and animal brains as a model for alignment between humans and AI systems

A moral backlash against AI will probably slow down AGI development

The heritability of human values: A behavior genetic critique of Shard Theory

Brain-over-body biases, and the embodied value problem in AI alignment

The heterogeneity of human value types: Implications for AI alignment