Goodhart's Law

Goodhart's Law is of particular relevance to AI Alignment. Suppose you have something which is generally a good proxy for "the stuff that humans care about", it would be dangerous to have a powerful AI optimize for the proxy, in accordance with Goodhart's law, the proxy will breakdown.  

• Groupthink, Information cascade, Affective death spiral
• Adaptation executers, Superstimulus
• Signaling, Filtered evidence
• Cached thought
• Modesty argument, Egalitarianism
• Rationalization, Dark arts
• Epistemic hygiene
• Scoring rule

Goodhart's lawLaw states that oncewhen a certain indicator of success is made aproxy for some value becomes the target of optimization pressure, the proxy will cease to be a social or economic policy, it will losegood proxy. Consider the information content that would qualify it to play suchSoviet story of a role.

Blog posts

• Markets are Anti-Inductive
• The Importancefactory graded on how many shoes they produced (a good proxy for productivity) – they soon began producing a higher number of tiny shoes. Useless, but the numbers look good.

  Goodhart's Law is of particular relevance to AI Alignment by blogospheroid

• Is Google Paperclipping. Suppose you have something which is generally a good proxy for "the stuff that humans care about", it would be dangerous to have a powerful AI optimize for the Web? The Perils of Optimization by Proxyproxy, in Social Systems by Alexandros

See also

• Groupthink, Information cascade, Affective death spiral
• Adaptation executers, Superstimulus
• Signaling, Filtered evidence
• Cached thought
• Modesty argument, Egalitarianism
• Rationalization, Dark arts
• Epistemic hygiene
• Scoring rule

See Also

• Groupthink, Information cascade, Affective death spiral
• Adaptation executers, Superstimulus
• Signaling, Filtered evidence
• Cached thought
• Modesty argument, Egalitarianism
• Rationalization, Dark arts
• Epistemic hygiene
• Scoring rule

Goodhart's Law states that when a proxy for some value becomes the target of optimization pressure, the proxy will cease to be a good proxy. ConsiderOne form of Goodhart is demonstrated by the Soviet story of a factory graded on how many shoes they produced (a good proxy for productivity) – they soon began producing a higher number of tiny shoes. Useless, but the numbers look good.

Goodhart Taxonomy

In Goodhart Taxonomy, Scott Garrabrant identified four kinds of Goodharting:

• Regressional Goodhart - When selecting for a proxy measure, you select not only for the true goal, but also for the difference between the proxy and the goal.
• Causal Goodhart - When there is a non-causal correlation between the proxy and the goal, intervening on the proxy may fail to intervene on the goal.
• Extremal Goodhart - Worlds in which the proxy takes an extreme value may be very different from the ordinary worlds in which the correlation between the proxy and the goal was observed.
• Adversarial Goodhart - When you optimize for a proxy, you provide an incentive for adversaries to correlate their goal with your proxy, thus destroying the correlation with your goal.

In Goodhart Taxonomy,Taxonomy, Scott Garrabrant identified four kinds of Goodharting:

In Goodhart Taxonomy, Scott Garrabrant identifiedidentifies four kinds of Goodharting:

• Groupthink, Information cascade, Affective death spiral
• Adaptation executers, Superstimulus
• Signaling, Filtered evidence
• Cached thought
• Modesty argument, Egalitarianism
• Rationalization, Dark arts
• Epistemic hygiene