This article introduces Goodhart's law, provides a few examples, tries to explain an origin for the law and lists out a few general mitigations.

Goodhart's law states that once a social or economic measure is turned into a target for policy, it will lose any information content that had qualified it to play such a role in the first place. wikipedia The law was named for its developer, Charles Goodhart, a chief economic advisor to the Bank of England.

The much more famous Lucas critique is a relatively specific formulation of the same. 

The most famous examples of Goodhart's law should be the soviet factories which when given targets on the basis of numbers of nails produced many tiny useless nails and when given targets on basis of weight produced a few giant nails. Numbers and weight both correlated well in a pre-central plan scenario. After they are made targets (in different times and periods), they lose that value.

We laugh at such ridiculous stories, because our societies are generally much better run than Soviet Russia. But the key with Goodhart's law is that it is applicable at every level. The japanese countryside is apparently full of constructions that are going on because constructions once started in recession era are getting to be almost impossible to stop. Our society centres around money, which is supposed to be a relatively good measure of reified human effort. But many unscruplous institutions have got rich by pursuing money in many ways that people would find extremely difficult to place as value-adding.

Recently GDP Fetishism by David henderson is another good article on how Goodhart's law is affecting societies.

The way I look at Goodhart's law is Guess the teacher's password writ large. People and instituitions try to achieve their explicitly stated targets in the easiest way possible, often obeying the letter of the law. 

A speculative origin of Goodhart's law

The way I see Goodhart's law work, or a target's utility break down, is the following.

  • Superiors want an undefined goal G.
  • They formulate G* which is not G, but until now in usual practice, G and G* have correlated.
  • Subordinates are given the target G*.
  • The well-intentioned subordinate may recognise G and suggest G** as a substitute, but such people are relatively few and far inbetween. Most people try to achieve G*. 
  • As time goes on, every means of achieving G* is sought. 
  • Remember that G* was formulated precisely because it is simple and more explicit than G. Hence, the persons, processes and organizations which aim at maximising G* achieve competitive advantage over those trying to juggle both G* and G. 
  • P(G|G*) reduces with time and after a point, the correlation completely breaks down.

The mitigations to Goodhart's law

If you consider the law to be true, solutions to Goodhart's law are an impossibility in a non-singleton scenario. So let's consider mitigations.

  • Hansonian Cynicism
  • Better Measures
  • Solutions centred around Human Discretion

Hansonian Cynicism

Pointing out what most people would have in mind as G and showing that institutions all around are not following G, but their own convoluted G*s. Hansonian cynicism is definitely the second step to mitigation in many many cases (Knowing about Goodhart's law is the first). Most people expect universities to be about education and hospitals to be about health. Pointing out that they aren't doing what they are supposed to be doing creates a huge cognitive dissonance in the thinking person.

Better measures

Balanced scorecards

Taking multiple factors into consideration, trying to make G* as strong and spoof-proof as possible.  The Scorecard approach is mathematically, the simplest solution that strikes a mind when confronted with Goodhart's law.

Optimization around the constraint

There are no generic solutions to bridging the gap between G and G*, but the body of knowledge of theory of constraints is a very good starting point for formulating better measures for corporates.

Extrapolated Volition

CEV tries to mitigate Goodhart's law in a better way than mechanical measures by trying to create a complete map of human morality. If G is defined fully, there is no need for a G*. CEV tries to do it for all humanity, but as an example, individual extrapolated volition should be enough. The attempt is incomplete as of now, but it is promising.

Solutions centred around Human discretion

Human discretion is the one thing that can presently beat Goodhart's law because the constant checking and rechecking that G and G* match. Nobody will attempt to pull off anything as weird as the large nails in such a scenario. However, this is not scalable in a strict sense because of the added testing and quality control requirements.

Left Anarchist ideas

Left anarchist ideas about small firms and workgroups are based on the fact that hierarchy will inevitably introduce goodhart's law related problems and thus the best groups are small ones doing simple things.

Hierarchical rule

On the other end of the political spectrum, Molbuggian hierarchical rule completely eliminates the mechanical aspects of the law. There is no letter of the law, its all spirit. I am supposed to take total care of my slaves and have total obedience to my master. The scalability is ensured through hierarchy.


Of all proposed solutions to the Goodhart's law problem confronted, I like CEV the most, but that is probably a reflection on me more than anything, wanting a relatively scalable and automated solution. I'm not sure whether the human discretion supporting people are really correct in this matter.

Your comments are invited and other mitigations and solutions to Goodhart's law are also invited.

Moderation Guidelinesexpand_more

There are no generic solutions to bridging the gap between G and G*, but the body of knowledge of theory of constraints is a very good starting point for formulating better measures for corporates.

A good example from my own history of doing this is when I worked for an ISP and persuaded them to eliminate "cases closed" as a performance measurement for customer service and tech support people, because it was causing email-based cases to be closed without any actual investigation. People would email back and create a new case, and then a rep would get credit for closing that one without investigation either.

The replacement metric was one I derived via the Theory of Constraints, inspired by Goldratt's "throughput-dollar-days" measurement. The replacement metric was "customer-satisfaction-waiting-hours" - a measurement of collective work-in-progress inventory at the team level, and a measurement of priority at the ticket level.

I also made it impossible to truly "close" a case - you could say, "I think this is done", but the customer could still email into it and it would jump right back to its old place in the queue, due to the accumulated "satisfaction waiting hours" on the ticket.

Of course, the toughest part in some ways was educating new service managers that, no, you can't have a measurement of cases closed on a per-rep basis. Instead, you're going to have to actually pay attention to a rep's work in order to know if they're doing the job. (Of course, the system I developed also had ways to make it easy to see what people are working on, not only at the managerial but the team level - peer pressure is a useful co-ordination tool, if done right.)

I have no idea how well the system fared since I left the company, since it's entirely possible they found programmers since then to give them new metrics that would f**k it up, although I did design the database in such a way as to make it as close to impossible as I could manage. ;-)

Anyway, the theory of constraints positively rocks for business performance optimization, and its Thinking Processes are generally useful tools for any rationalist. They were also a big inspiration for me developing other thinking processes and ultimately mindhacking techniques, in that they showed that it's possible to think systematically even about some of the vaguest and most ill-defined problems imaginable, rigorously hone in on key leverage points, resolve conflicts between goals, and generally overcome our brains' processing limitations for analysis and planning.

[Edit to add: the Wikipedia page on thinking processes doesn't really show why a rationalist would be interested in the processes; it's useful to know that a key element of the processes are something called the "categories of legitimate reservation", which have to do with logical proof and well-formedness of argument. They are a key part of constructing and critiquing the semantic maps that are created by the thinking processes.

For example, ToC's conflict resolution method effectively maps out certain implicit assumptions in a conflict, and then invites you to logically disprove these assumptions in order to break the conflict. (That is, if you can find a circumstance where one of those assumptions is false, then the conflict will no longer exist under that circumstance - and you have a potential way out of your dilemma.)

So, in short, ToC thinking processes are mostly about constructing past, present, or future semantic maps of a situation, and applying systematic logic to validating (or invalidating) the maps' well-formedness, as a way of solving problems, creating plans, etc. Very core rationalist stuff, from an instrumental-rationality POV.]

I am reminded of one of Dijkstra's sayings:

To this very day we have organizations that measure "programmer productivity" by the "number of lines of code produced per month"; this number can, indeed, be counted, but they are booking it on the wrong side of the ledger, for we should talk about "the number of lines of code spent".

So, in short: incentives can have unintented consequences, as the incentives influence whatever you want to influence with them.

There are a lot of examples of this in e.g. Dan Ariely's book and Freakonomics.

But the best example must be the bizarre 1994 footbal (soccer) match between Barbados and Grenada. Barbados needed to win with a two goal difference.

The special incentive here was that any goal scored in the extra time would count double. Now, shortly before the end of the regular time, it was 2-1 for Barbados. Imagine what happened...

(edit: added the note about the two-goal difference, thanks Hook)

It's an important note for the soccer game that Barbados needed to win by two points in order to advance to the finals. Otherwise, Grenada would go to the finals. Now people have a chance of imagining what happened.

Goodhart's Law starts some other way. It's not quite right to say:

Superiors want an undefined goal G.

Mathematically speaking, the problem can't be that G is undefined. If G were really undefined in any absolute sense, then superiors would be indifferent to all possible outcomes, or would choose their utility function literally at random. That rarely happens.

Instead, the problem could be that G is difficult to articulate. It is "undefined" only in the sense that people have had trouble coming up with an explicit verbal definition for it. i know what I want and how to get it, but I don't know how to communicate that want to you ex ante. For example, maybe I want you (the night shift manager) to page me (the owner) whenever there's a decision to make that could affect whether our business keeps a client, but I've never taken any business classes and don't quite have the vocab to say that, so instead I say to only page me if it's "important." "Important" is vague, but "important' is just a map, and the map is not the territory.

Alternatively, the problem could be that G is difficult to commit to. I can define my goal in words just fine today, but I know (or you suspect) that later I will be tempted to evaluate you by some other criterion. For example, I would like to give a raise to whichever police officer does the most to keep his beat safe, and, as a thoughtful and experienced police chief, I know exactly what the difference is between a safe neighborhood and an unsafe neighborhood, and I'm happy to explain it to anyone who's interested. As one of my employees, though, you can't verify that I'm actually rewarding people for making neighborhoods safe, and not, say, giving raises to people who bring in the most money for drug busts, or who artificially lower their crime statistics, or who give me a kickback. It might make more sense for me to just announce that I'll pay people based on hours worked and complaints lodged, because that announcement is more verifiable, and thus more credible, so at least I'll be viewed as evenhanded.

Finally, as you've already pointed out, the problem could be that G is difficult or expensive to measure. Alternative measures of GDP that take into account factors like health, leisure, and environmental quality have gotten pretty good about specifying what health is, and it's easy enough to pass laws that commit agencies to valuing health in a particular way, but it's expensive to measure health, especially in any broad sense. A physical is $60; an exercise fitness exam is another $45; an STD test runs about $20; a battery of prophylactic tests for cancer and heart disease and so on is another $100 or so; a mental health exam is another $80, and then you multiply all that by the size of a valid random sample and we're talking real money. In my opinion, it would be money very, very well spent, but one can understand why GDP - which can be measured just by asking the IRS for a copy of its tax receipts - is such a popular metric. It's cheap to use.

CEV, until designed and defined properly, is just a black box that everyone universally agrees is 'good', but has little else in term of defining features.

It's not so much a box, but a method of filling the box. We just haven't filled the box yet.

The fact that students who are motivated to get good scores in exams very often get better scores than students who are genuinely interested in the subject is probably also an application of Goodhart's Law?

Partially; but a lot of what is being tested is actually skills correlated with being good in exams - working hard, memorisation, bending youself to the rules, ability to learn skill sets even if you don't love them, gaming the system - rather than interest in the subject.

But those skills don't correlate with doing good science, or with good use of the subject of the exams in general, nearly so well, and they are easy to test in other ways.

Goodhart's law seems very applicable to natural selection: the Blind Idiot God wants creatures to have higher fitness (G), and so creates targets that are correlated with fitness in the ancestral habitat (e.g., pleasure-seeking and pain-avoidance (G*)). Once you get creatures that are self-aware (us), they figure out G-star, and start optimizing for that instead of G.


Getting back to trying to propose practical mitigation strategies for goodhart's law, I propose a fairly simple solution: Choose a G*, evaluate performance based on it, but KEEP IT SECRET. This of course wouldn't really work for national scale, GDP-esque kind of situations, but for corporate management situations it seems like it could work well enough. If only upper management knows what G* is, it becomes impossible to optimize for it, and everyone has to just keep working under the assumption they're being evaluated on G.

Taking it a step further, to hedge against employees eventually figuring out G* and surreptitiously optimizing for it, you could have a bounty on guessing G* - the first employee who figures out what the mystery metric G* really is gets a prize, and as soon as it's claimed, you switch to using G**

The hedge is absolutely necessary, elsewise, a manager will just tell subordinates what G* is in order to look impressive for managing a high-performing group.

Andrew Grove (of Intel fame) wrote a book, High Output Management, suggesting that management needs two opposing metrics to avoid this problem. For example, measure productivity and number of defects, and score people on the combined results.

BTW the large nail/little nail joke has a third part. Soviet management eventually got a clue and started measuring by the value of the nails produced... and the result was the world's first solid-gold-nail factory.

Presumably finding arbitrarily many basic G*'s will be hard. Two ideas for dealing with this: 1. Even if you only have finitely many and they're all known, you could select one at random each time there's a switch. 2. Each time there's a switch, select a somehow-random linear (or some other sort, if you like) combination of your basic G*'s. (That would make guessing it in the first place quite hard, actually...)

if management are doing that then are neglecting a powerful tool in their tool-kit, because announcing a G will surely cause G to fall, and experience says that to begin with a well-chosen G and G remain correlated (because many of the things to do to reduce G also reduce G). It is only over time that G* and G detach.

Pretty much every trick in organization design or management can be thought of as a partial solution to this problem. Listing "anarchy" or "absolute authority" explicitly on a short list of solutions is therefore a bit misleading.

At work a large part of my job involves choosing G , and I can report that Goodhart's Law is very powerful and readily observable.
Further : rational players in the workspace know full-well that management desire G, and the G
is not well-correlated with G, but nonethelss if they are rewarded on G*, then that's what they will focus on.

The best solution - in my experience - is mentioned in the post: the balanced scorecard. Define several measures G1 G2 G3 and G4 that are normally correlated with G. The correlation is then more persistent : if all four measures improve it is likely that G will improve.

G1 G2 G3 G4 may be presented as simulaneous measures, or if setting four measures in one go is too confusing for people trying to prioritise (the frwer the measures the more powerful) they can be sequential. IE If you hope to improve G over 2 years, then measure G1 for two quarters, then switch the measurement to G2 for the next two and so on. (obviously you don't tell people in advance). NB this approach can eb effective, but will make you very unpopular.

measure G1 for two quarters, then switch the measurement to G2 for the next two and so on. (obviously you don't tell people in advance).

Why obviously? Are you so afraid that people would do the right thing without immediate incentives?

I think I'd measure G1 first, but would tell in advance that next quarter we will measure that one of G1,G2,G3,... which will be most critical at the beginning of that quarter.

Goodhart's Law is a very nice corollary to the Snafu Principle: Communication is impossible in a hierarchy.

Temple Grandin has written about the importance of finding relevant, measurable standards-- the example she gives is the number of cattle falling down on the way to slaughter. Not falling down means the genes, food, lighting, walking surface etc. are all good enough.

Thing to check: Do measures used as targets for policy always become completely useless, or do they sometimes become increasingly less useful, but not totally useless? Does culture matter? I suspect that the amount of judgement which people are allowed to mix into the system varies a lot.

She seems to believe that thinking visually will be more likely to produce such standards than (as most people do) verbally.

I'm not convinced of that-- dog and cat show standards are an example of well-defined visual standards not producing reliably good results.

I have no idea if it's even possible to have that good a standard for financial markets.

I've heard "you can't manage what you can't measure", but I think "you can't manage what you can't perceive" is better. Is it possible to generalize the idea of the king traveling incognito to see how the kingdom is doing?

Once management recognizes that there is something to measure, I think they do an OK job measuring it - secret shoppers come to mind. But there's something more subtle about when you take for granted that G = G* and don't even think to verbalize your true values, so can't measure them.

The secret shoppers are a variant of "the king going incognito"-- but not as good in some ways because they may be tasked with evaluating according to a checklist, and thus could still be trapped by G vs G*.

I believe that the problem isn't that true values aren't verbalized, it's that they can't be fully verbalized. Language is too low-bandwidth to capture all the aspects of a situation.

The point of a king going incognito isn't just to enforce existing, verbalized rules, it's to see how things are in the kingdom. It's a bit easier for a king than an AI because a king is more like a subject than an AI is like people.

Does culture matter?

Yes. Culture is partly, a process of making people behave in a predictable fashion. If you make your subordinate as similar to you as possible, then there is a good chance that he/she will perceive G instead of G . But you have to be committed to juggling G and G and take the risk that someone actively pursuing G* is not getting ahead of you.

I wish there were some examples (other than the Soviet nails) ... if I had some better idea of what G and G* might actually represent, I'd be able to more easily get my head around the rest of the post.

In education, this is one of the criticisms of high-stakes testing: you'll just get schools teaching to the test, in ways that aren't correlated to real learning (the test is G*, real knowledge/learning is G). People say the same thing about the SAT and test prep - kids get into better colleges because they paid to learn tricks for answering multiple choice questions. The Wire does a great job of showing the police force's efforts to "juke the stats" (e.g. counting robberies as larcenies) so that crime statistics (G*) look better even while crime (G) is getting worse. Athletes get criticized for playing for their stats (G*), or trying to pad their stats, instead of playing to win, when the stats are supposed to be a measure of how much a player has contributed to his team's chances of winning (G). I'm not sure if it's historically accurate, but I've heard that body count (G*) was used by the US as one of the main metrics of success (G) in the Vietnam war, and as a result we ended up with a bunch of dead bodies but a misguided war.

In general, any time you measure something you care about in order to incentivize people, or to hold people accountable, or to keep track of what's going on, and the thing you measure isn't exactly the same as the thing that you care about, there's a risk of figuring out ways to improve the measurement that don't translate into improvements on the thing that you care about.

I'm surprised no one has yet brought up (G*) the LW karma system as a proxy for (G) contributing to "refining the art of human rationality".

LW karma is an interesting example because no one has direct access to the karma giving algorithm.

It's a bit like telling the nail factory that you're going to evaluate them on something, but not telling them whether its nail mass or number or something else until the end of the evaluation period.

If the one being evaluated knows nothing about how he's going to be evaluated except that it's going to be a proxy for goodness, then he can't really cheat. However, they might know that it's going to be very simple criteria so they make a very massive nail and many miniature ones.

This reminds me of the way I hear they do state censorship in China. The censoring agencies don't actually give out any specific guidelines on what is allowed and what isn't, instead just clamping down on cases they do consider to be over the line. As a consequence, everyone self-censors more than they might with specific guidelines: with the guidelines, you could always try to twist their letter to violate their spirit. Instead, people are constantly unsure of just exactly what will get you in trouble, so they err on the side of caution.

While I strongly oppose state censorship, I can't help but admire the genius in the system.

Also, unlike Saudi Arabia, they don't make many efforts to block pornography. As a result, the average Chinese teen is less likely to know how to access blocked sites than the average Saudi teen is (or so I read; I'm not aware of any study on that).

Depressing. This would mean that most informal norms of censorship are much more resilient and effective than most formal laws censoring material.

Arguably this makes them much harder to dislodge than even the intentionally vague Chinese law. Since I guess you can't really be prosecuted under it by pointing out there is a censorship law right?

Or section 28 , which didn't forbid the discussion of homosexuality in the classroom, only its promotion....but since promotion wasn't defined, schools erred on the side of not mentioning it.

What is your interpretation of it? It seems a pretty plausible hypothesis to me that it's a proxy for something, and has come to be relied upon as such. If we think Goodhart's Law applies in the case of karma, the final prediction in the "speculative origin" section might be something to be concerned about.

I think of it as a proxy for "valued member of the community" - if someone has karma, then people like their posts and comments. I'm mostly here to have fun and pass the time, and I happen to find discussing rationality to be fun. I don't really expect refining the art of human rationality to be well-correlated with a popularity contest.

And do you think Goodhart's Law, as presented in the post, applies here? That is, we should expect that eventually people (through gaming the system) end up with high karma without that in fact reliably correlating with being valued members of the community?

As a data point, one thing I've noticed that seems to give a disproportionate amount of karma is arguing with someone who's wrong and unwilling to listen. It's easy to think they might come around eventually, and each point you make against them is worth a few points of karma from the amused onlookers or fellow arguers - which might tell you that you're making a valuable contribution, and so encourage you to keep arguing with trolls. This is my impression, at least.

Edit: (The problem being - determining the point of diminishing returns.)

Except we're like the self-employed in this regard. You can't do anything with karma. It won't impress your boss. It is just a way of quantifying how valued you are by the community. An employee doesn't really care about G at all. She cares about G because that's what impresses the boss which furthers her own goals. But if you are your own boss you do care about G, G is just an easy way to measure it. For me at least, this is the case with karma. I can't do anything with the number but it suggests that people like me.

So perhaps revenue sharing is a way to help address the problem. Instead of trying to come up with ways to measure what you care about, make the people beneath you care about it too. Of course this is a lot easier with money than it is with values.

Only if people care about having high karma. It's probably fairly easy to game karma by making multiple accounts and voting yourself up, but why bother?

And do you think Goodhart's Law, as presented in the post, applies here? That is, we should expect that eventually people (through gaming the system) end up with high karma without that in fact reliably correlating with being valued members of the community?

What? You mean Karma doesn't reliably correlate with objective worth of the individual? Damn.

The health and/or beauty of a woman (G) and her scale reported weight (G*) which might be somewhat correlated under some circumstances, but are definitely not identical and can diverge rather sharply due to crazy diets.

Call time (G) or calls taken (G) in a call center, where what they care about is customer satisfaction (G) (at least inasmuch as it serves profitability).

I noticed this tendency in British running of hospitals, schools and police forces. The gov got hooked on the idea of targets and not on medicine, education and public order.

And yet, correlation between government targets and results is probably a lot higher than correlation between teachers/doctors/policemen fuzzy ideas how their job should be done and results.

Maybe, but how probably and how much? Why do you think that? For which governments? By what measure?

1) Examples of G* should be given a cost-benefit analysis. Yeah, scammers and parasites exist, but societies that use money still seem to better off than societies that try to get rid of it.

2) It's unclear to me why you list CEV as one of the solutions. We use money to allocate limited resources. If magic nano-AI appears and resources become unlimited, why keep score at all? If it doesn't and resources stay limited, how does CEV help you distribute bread, and would you really like it to replace money? (I wouldn't. No caring daddies for me, please.)

In the case of a FAI G would be friendliness and G the friendliness definition. Avoiding a Goodhart's Law effect on G is pretty much the core of the friendliness problem in a nutshell. An example of such a Goodhart's Law effect would be the molecular smiley faces scenario.

Ah, sorry. I've read the post as saying something different from what it actually says.

Good discussion.

The point I wanted to make was about Extrapolated volition as a strategy to avoid Goodhart's law issues. If you extrapolate the volition of a person towards the "person he/she wants to be" and put a resulting goal as G*, it will be pretty much close to G as can be. I presented CEV as an example, since the audience is more familiar with it.

And FAWS, your definition of G and G* in the friendliness scenario is perfect. I've nothing more to add there.

Often a goal set is not based on a single set of arguments justifying it, but because it is a good compromise point between multiple arguments, motivations or interest groups. For example human rights formulations don't perfectly fulfill any groups desires (utilitarians, egalitarians, deontological groups, religious motivations etc.) but are a point of overlap between their goal sets (both utilitarians and deontologists both think torture and murder are generally bad). Similarly with GDP, economic growth is a shared interest of several groups in society.

So some instances of goodhearts law may be an observation that particular sets of goals are not being perfectly fulfilled.

I suggest editing a "summary break" into this post to create a "continue reading" link on the frontpage. It's the 6th button from the left atop the editing interface.


The Big Mac Index has been used to compare prices across countries, as we have noted before. Argentina currently has very high prices due to a combination of inflation and a strong economy, and this shows up glaringly in the Big Mac Index.

Tyler Cowen reports (translating a Spanish original) that the Argentinian government has persuaded McDonalds to lower the price of the Big Mac (relative to other McDonalds items, and relative to competing hamburgers), so that Brazil’s Big Mac Index becomes more competitive.

In other words, the real price of the Big Mac rose nearly twice as much as the official statistics were willing to admit, in Argentina of course. That’s not right, so the government sprang into action. The minister of the commerce department “persuaded” McDonald’s to price the Big Mac at $16, while other sandwiches at the chain are in the $21 to $23 range.

The outlets now keep the Big Mac well-hidden

body of knowledge of theory of constraints is a very good starting point for formulating better measures for corporates

I've had some interest in TOC, could you please expand on how it works to get G* closer to G?

Generally I've found TOC to be some really interesting semi-scientific stuff mixed with a ton of self promotion by goldratt.

One method is to have no G*. Tell people some of the things you'll be looking at, but don't give them specific targets or tell them exactly how you will be judging G.

This is the method currently in use to assess the quality of research in departments of universities in the UK. Every department that wants to be assessed must supply certain very detailed information (e.g. for each member of staff declared as doing research, a list of their recent publications, grants held, awards received, etc.). The actual assessment is carried out behind closed doors. They give general guidelines about their criteria, but the process is one of "expert review, informed by indicators where appropriate".

This is done every few years. The data requested changes every time, and sometimes even the name of the exercise. As government funding depends on the outcome, every research-active department is desperate to get a good rating.

The same issue came up many years ago in the work of H Edwards Deming in quality control.

  • Any data used to reward and/or punish people will become useless for managing the organization.

I like this article a lot. My solution (borrowed from Nassim Taleb) would be skin in the game. Any potential outcome resulting from the actions of the agent should be also affecting the agent.

Interesting: Left Anarchist, Right Libertarian, and Distributist ideals are fundamentally the same. While Right-Libertarians pay a form of lip service to the idea of hierarchical corporate capitalism, scratch them a bit and you find they long for SV startups or farmers on the American Frontier, as presented in books like The Moon Is A Harsh Mistress - family businesses or small, egalitarian workgroups like the 3 guys who founded YouTube. And Left-Anarchism and Disrtributism are pretty much the same, the difference is LA putting a mainly socially liberal sauce on it, while Distributism putting a socially reactionary medievalist-catholic sauce on it (medieval artisans loosely cooperating in guilds being their ideal). <