Heuristics and Biases are the ways human reasoning differs from a theoretical ideal agent, due to reasoning shortcuts that don't always work (heuristics) and systematic errors (biases).

See also: Affect Heuristic, Confirmation Bias, Fallacies, Predictably Wrong, Rationality, Your Intuitions Are Not Magic

Basics

“Cognitive biases” are those obstacles to truth which are produced, not by the cost of information, nor by limited computing power, but by the shape of our own mental machinery. For example, our mental processes might be evolutionarily adapted to specifically believe some things that arent true, so that we could win political arguments in a tribal context. Or the mental machinery might be adapted not to particularly care whether something is true, such as when we feel the urge to believe what others believe to get along socially. Or the bias may be a side-effect of a useful reasoning heuristic. The availability heuristic is not itself a bias, but it gives rise to them; the machinery uses an algorithm (give things more evidential weight if they come to mind more readily) that does some good cognitive work but also produces systematic errors.

Our brains are doing something wrong, and after a lot of experimentation and/or heavy thinking, someone identifies the problem verbally and concretely; then we call it a “(cognitive) bias.” Not to be confused with the colloquial “that person is biased,” which just means “that person has a skewed or prejudiced attitude toward something.”

A bias is an obstacle to our goal of obtaining truth, and thus in our way.

We are here to pursue the great human quest for truth: for we have desperate need of the knowledge, and besides, we're curious. To this end let us strive to overcome whatever obstacles lie in our way, whether we call them “biases” or not.

It's also useful to know the kinds of faults human brains are prone to, in the same way it's useful to know that your car's brakes are a little gummy (so you don't sail through a red light and into an 18-wheeler).

The Sequence, Predictably Wrong, offers an excellent introduction to the topic for those who are not familiar.

Wait a minute... fallacies, biases, heuristics... what's the difference??

While a bias is always wrong, a heuristic is just a shortcut which may or may not give...

Thingspace