This post was finished for Good Heart Week and speaks to perhaps the most typical context for Goodharting.
Context: Theory of Money
Money is one of the largest factors in determining what gets done in the world and a clearer understanding of it and how it interacts with what we value may pay dividend$.
It is commonly claimed that money is an estimate of value and some seem to even think that it is value. While clearly this isn’t entirely true, let's examine it in more detail, and see what money really is and what it really tracks.
What is money? At first glance it seems to be something you can convert into arbitrary goods and services by paying for things. In that sense it seems kind of similar to, say, having iron ore that you may convert into nails. It’s a bit more subtle though. Firstly it’s something you may convert into practically anything, a kind of universal resource. For another, it isn’t itself generally destroyed in the process and is instead simply exchanged: money moves towards the person giving the good or service and away from the person receiving the good or service. It is a kind of resource that can be created from nothing or destroyed and be of any number from positive to negative infinity. Finally, it is supported by a kind of universal belief and convention; if you fail to believe in it, then it ceases to exist, and the belief that exists is distributed amongst all the people involved in transactions. Given how it works I would say it is the natural generalization of an IOU or social status change and probably originated in that (History of money).
From this then it’s reasonable to say that there are five significant operations on money:
- Getting it in exchange for goods or services
- Giving it to receive goods or services
- Storing it for future use like you may most resources
- Creating it
- Deleting it
Giving these operations, it then looks like money is a kind of anti resource that you exchange for other resources; when you get a good or service that act annihilates a certain amount of money for you. In some derivative-like sense it is the opposite of the change in what you got in exchange.
How do these money calculations then ground out so you get concrete prices? They ground out in the decisions of people to exchange it for things they value intrinsically, the brute facts of resource conversion efficiency, and in the short run (assuming proper updating) beliefs about resource conversion efficiency.
With these clarifications in the background we can now go on and see how money differs from the intrinsic value of things and how it fails to track that exactly.
What does money track?
- Utility of goods or services, sometimes money really is tracking raw utility of options. When they say they’d prefer to get two watermelons for the amount they’d be paid for three apples this expresses that intrinsic value (ignoring for simplicity the distinction between terminal and instrumental values (see supergoal uncertainty, moral uncertainty sequence, and moral circles)). This also easily incorporates risk seeking or risk avoiding preferences.
- Scarcity or effort to create things or do actions and rivalrous goods and services, when it takes a lot of time, people, pain,... etc. to produce something that difficulty is represented as a higher price. In addition, the price is dependent upon if the thing just exists for a limited time, whether there are resource thresholds for producing it, overheads of coordination between buyers and sellers, and the existence of economies of scale. This doesn’t mean it is in some sense a good thing to produce, it just means that it is scarce or hard to produce.
- Beliefs about effort or utility, sometimes we store a resource to use another day, speculatively value a stock, revalue a currency, discount future resources, or accidentally think a product will be worth more then it turns out to. Because agents are fundamentally deciding what exchanges they are willing to accept, they determine how much money each thing is worth, and in an uncertain world those prices involve beliefs, right or wrong.
- Historical accidents, as with many of society’s beliefs a lot is determined by simple accidental incidental beliefs or conventions. Whether we value rubies or diamonds more, prefer private or public transport, or value one art piece over another can be highly historically contingent. Sometimes what processes of resource conversion are discovered and what piles of resources are discovered in the wild may also be highly random. Also, sometimes in thin markets (those with few customers and suppliers) there can be a good deal of noise in value assignment and what agents exist can depend a lot on chance. Finally, people can sometimes make money on guesses that turn out to be incorrect eventually but convert that money to other assets before correction happens and thus they avoid the valuation update.
- Negotiating power, game theory, and excludable goods or services, many agents have partial bottleneck access to options to choose amongst and sometimes an agent can intentionally inflate the difficulty of producing something or subsidize the usefulness of something and so produce systematic changes in prices for goods. They can segment the market, rent seek and use barriers to entry, use threats and force, undercut and outlast their competition,....
- Relative resources of types of agents, if some agents have a lot more resources than others then their beliefs and intrinsic values tend to distort valuations in their direction. By means of competition for resources and the production of more of certain types of agents this distortion will vary in time. There are some statistical tendencies in what agents get resources (for example evolution, thermodynamics, Moloch and also movements along the production frontier) but there is also a lot of randomness (relatedly).
- Coordination, Schelling points, societal decision power and deference, sometimes money can be used as a message or signal to promote coordination and deference for societal decision making. This is one of the primary functions of central banking.
- The quantities set by those who can produce or delete money or more exotic transformations, for many currencies there are some systems that can produce or delete money leading to inflation or deflation of the underlying currency and addition or subtraction of value for those nearby (where production or deletion is happening) in the trade network. Furthermore, in theory arbitrary other exotic transformations and rules can be added to the monetary system causing a distortion (and possible loss of trust in and deflation of the currency).
- Direct effects, generally not externalities, much of the time pricing doesn’t track side-effects and externalities though with less bounded rationality, market places, and the observations of Coase these can sometimes be accounted for.
- Non-agentful or non-updating causes of trade and the relative resources of these systems, sometimes people or systems will act in non-agentful ways and engage in trade for reasons other than maximizing utility based on beliefs. They or it (say an automated trading system) could be going with a fad, reacting emotionally, be bounded rational, copying others, or executing arbitrary programs that cause trades.
Using this
Knowing how money fails to track value and refining our understanding of how it works could possibly help us in multiple ways:
- Indicate ways to make additional money for EA and x-risk cause areas
- Remind us that the market doesn’t perfectly track value and that what is valuable isn’t the same as what is expensive and that who contributes most to society isn’t the same as who is rich.
- Develop ways to model money’s deviation from tracking value so we can more clearly figure out what people value based on what they are willing to pay (possibly useful for AI Alignment. See also 1, 2)
- Point towards ways to improve the monetary system and get it more aligned with EA, if the ways in which the monetary system is flawed can be fixed so that it more closely follows value. For instance, if the beliefs parameter changed so that the majority of people cared about longtermism, then automatically longtermist work would become directly profitable and wouldn’t need to be funded by donations. Perhaps we eventually would value things in a CEV fashion and can accelerate that process.
- Suggest replacements for the idea of money that track things more accurately (though be aware of Goodharting and Gresham’s law. Maybe what we need is a more complex idea that echos reality closer at multiple levels of approximation? Taylor expansion dollars? Complex number valued currency? Hyperbolic discounting coin? Implementations of prediction markets and impact certificates?
- Use the near universal belief in money as precedence for and inspiration for changing the world in a large way like actually figuring out the ground truth of morality and getting consensus around it.