Thanks for the post. I love it.
My comments:
First sidenote that dont assume that if something is a heuristic it is automatically a wrong way of thinking.(sorry if i misinterpret this, because you dont explicitly say this at all :) In some situations simple heuristics will outperform regression analysis for example.
But about your mainpoint. If I understood right this is actually a problem of violating so called "ratio rule".
(1) The degree to which c is representative of S is indicated by the conditional propability p (c | S)- that is, the propability of members of S have characterestic c.
(2) The propability that the characteristic c implies membership S is given by p (S | c). (Like you write)
(3) p (c | S) / p (S | c) = p(c) / p(S)
This is the Ratio Rule= Ratio of inverse propabilities equals the ratio of simple propabilities. So to equate these two propabilities p(c|S) and p(S|c) in the absence of equating ALSO the simple propabilitis is just wrong and bad thinking.
Representative thinking does not reflect these differences between p(c|S) and p(S|c) and introduces a symmetry in the map (thought) that does not exist in the world.
For example: "Home is the most dangerous place in the world because most accidents happen in home. So stay away from home!!!" --> This is confusion about the propability of accident given being home with propability being home given accident.
Thank you. English isn't my first language, so for me feedback means a lot. Especially positive :)
My point was that representative heuristic made two errors: firstly, it violates "ratio rule" (= equates P(S|c) and P(c|S)), and secondly, sometimes it replaces P(c|S) with something else. That means that the popular idea "well, just treat it as P(c|S) instead of P(S|c); if you add P(c|~S) and P(S), then everything will be OK " doesn't always work.
The main point of our disagreement seem to be this:
(1) The degree to which c is representative of S is indicated by the conditional propability p (c | S)- that is, the propability of members of S have characterestic c.
1) Think about stereotypes. They are "represent" their classes well, yet it's extremely unlikely to actually meet the Platonic Ideal of Jew.
(also, sometimes there is some incentive for members of ethnic group to hide their lineage; if so, then P(stereotypical characteristics|member of group) is extremely low, yet the degree of resemblance is very high)
(this is somewhat reminds me of the section about Planning Fallacy in my earlier post).
2) I think that it can be argued that the degree of resemblance should involve P(c|~S) in some way. If it's very low, then c is very representative of S, even if P(c|S) isn't high.
Overall, inferential distances got me this time; I'm probably going to rewrite this post. If you have some ideas about how this text could be improved, I will be glad to hear them.
Just to clarify, I feel that what you're basically saying that often what is called the base-rate fallacy is actually the result of P(E|!H) being too high.
I believe this is why Bayesians usually talk not in terms of P(H|E) but instead use Bayes Factors.
Basically, to determine how strongly ufo-sightings imply ufos, don't look at P(ufos | ufo-sightings). Instead, look at P(ufos | ufo-sightings) / P(no-ufos | ufo-sightings).
This ratio is the Bayes factor.
Thank you for your feedback.
Yes, I'm aware of likelihood ratios (and they're awesome, especially for log-odds). Earlier draft of this post ended at "the correct method for answering this query involves imagining world-where-H-is-true, imagining world-where-H-is-false and comparing the frequency of E between them", but I decided against it. And well, if some process involves X and Y, then it is correct (but maybe misleading) to say that in involves just X.
My point was that "what it does resemble?" (process where you go E -> H) was fundamentally different from "how likely is that?" (process where you go H -> E). If you calculate likelihood ratio using the-degree-of-resemblance instead of actual P(E|H) you will get wrong answer.
(Or maybe thinking about likelihood ratios will force you to snap out of representativeness heuristic, but I'm far from sure about it)
I think that I misjudged the level of my audience (this post is an expansion of /r/HPMOR/ comment) and hadn't made my point (that probabilistic thinking is more correct when you go H->E instead of vice versa) visible enough. Also, I was going to blog about likelihood ratios later (in terms of H->E and !H->E) — so again, wrong audience.
I now see some ways in which my post is debacle, and maybe it makes sense to completely rewrite it. So thank you for your feedback again.
(x-posted from my blog)