No nonsense version of the "racial algorithm bias"
In discussions of algorithm bias, the COMPAS scandal has been too often quoted out of context. This post gives the facts, and the interpretation, as quickly as possible. See this for details. THE FIGHT The COMPAS system is a statistical decision algorithm trained on past statistical data on American convicts. It takes as inputs features about the convict and outputs a "risk score" that indicates how likely the convict would reoffend if released. In 2016, ProPublica organization claimed that COMPAS is clearly unfair for blacks in one way. Northpointe replied that it is approximately fair in another way. ProPublica rebukes with many statistical details that I didn't read. The basic paradox at the heart of the contention is very simple and is not a simple "machines are biased because it learns from history and history is biased". It's just that there are many kinds of fairness, each may sound reasonable, but they are not compatible in realistic circumstances. Northpointe chose one and ProPublica chose another. THE MATH The actual COMPAS gives a risk score from 1-10, but there's no need. Consider the toy example where we have a decider (COMPAS, a jury, or a judge) judging whether a group of convicts would reoffend or not. How well the decider is doing can be measured in at least three ways: * False negative rate = (false negative)/(actual positive) * False positive rate = (false positive)/(actual negative) * Calibration = (true positive)/(test positive) A good decider should have false negative rate close to 0, false positive rate close to 0, and calibration close to 1. Visually, we can draw a "square" with four blocks: * false negative rate = the "height" of the false negative block, * false positive rate = the "height" of the false positive block, * calibration = (true positive block)/(total area of the yellow blocks) Now consider black convicts and white convicts. Now we have two squares. Since they have different reoffend rates for some reason, the