This is a linkpost for https://brianlui.dog/2018/03/03/correct-models-are-bad/
I think the claims here are fine but the title is a bit clickbaity/metacontrian in a way I'm not a huge fan of.
No. Correct models are good. Or rather, more correct models, applied properly, are better than less correct models, or models applied wrongly.
All those examples, however, are bad:
Calories in / Calories out is a bad model because different sources of calories are metabolized differently and have different effects on the organism. It is bad because it is incomplete and used improperly for things that it is bad at. It stays true that to get output from a mechanism, you do have to input some fuel into it; CICO is good enough to calculate, for example, how many calories one should eat in, say polar climates where one needs many thousands of them to survive the cold; that is not an argument against using a Correct model, it is an argument for using a model that outputs the correct type of data for the problem domain.
Being unwilling to update is bad, yes, that is a problem with the user of the model, not a problem with the model one is using. Do you mean that knowing and using the best known model makes one unwilling to update on the subsequent better model? Because that is still not a problem with using a Correct model.
That is an entirely different definition of Bad. "Bad for the person holding the model (provided they can't pretend to hold the more socially-accepted model when that is more relevant to staying alive and non-ostracized)" is absolutely not the same thing as "Bad at actually predicting how reality will react on inputs".
That is also a problem with the users of the model, both those making the prediction and those accepting the predictions, in domains that the model is not good at predicting.
Still not a problem due to using a Correct model. Also, bad outcomes for who? If immigration is good for the immigrants in some respsect and bad for them in some other respect, that is neither all good or all bad; if it is good for some citizens already there in some ways but bad in others (with some things being a lot of good for some citizens and some other things being a little bad, while also some things being a little good and other things being a lot of bad for other citizens), then the Correct model is the one that takes into account the entirety of all the distributions of the good and the bad things, across all citizens, and predict them Correctly.
( and now please in the name of Truth let this thread not devolve into an object-level discussion of that specific example >_< )
I think the title is a little bit misleading, and perhaps he didn't put much emphasis on this, but it seems he isn't claiming correct models are generally bad, just that there are also possible downsides to holding correct models and it's probably a good idea to be aware to these flaws when applying these models to reality.
Also, it seems to me as he is defining 'correct model' as a model in which the reasoning is sound and can be used for some applications, however does not necessarily fully describe every aspect of the problem.
Wrong models can also be ineffective, an obstacle to exploration, politically dangerous, easy to misuse, etc.
Agree with c0rwin.
You seem to be using to word "correct" to do a lot of work and different work in different cases.
CICO is useful to keep around despite being unhelpful as a reduction, because it can form part of a bigger model. In fact most dieting models rely on it and build around it.
this seems like an incorrect model that you "think is correct".
I'd call this a limited model. It's may be correct in the context of objectively what's happening but unsafe to bring up in the sociopolitical dinner party conversation.
"The model is easy to misuse" is a useful idea to be included with models that are easy to misuse. For example "growth mindset" model can be misused by justifying why you shouldn't try (because you have never been good at it and why start now). Knowing it can be misused is part of the model.
Unclear. I'd be taking the model and making a more detailed one including what happens with misuse.
I think a concrete definition of "correct" would help a lot here. All models are incomplete, so there are always phenomena that a model doesn't predict well. If you say "models that are useful for some things can be harmful when applied outside of those things", I think most controversy will dissolve.
Quick Summary:
We may think that correct mental models are always good to have, but the reality is that some correct models are useless or even harmful.
All models are simplifications of the world. Whenever we knowingly use a wrong model, we are aware of what the model doesn’t cover. The danger of using correct models is that we are more likely to apply the model without thinking about what’s been left out.
Some ways a correct model can be harmful:
1) The model is ineffective - Some correct models don’t seem to serve much purpose, so even though they are correct, it’s not really worth keeping them around. (ex: Calories In Calories Out)
2) The model is an obstacle to exploration - Once our brains have decided that a mental model is correct, we are unlikely to question the model. This can stunt our learning and personal growth. (ex: Initial research on overconfidence)
3) The model is socially or politically dangerous - Some correct models may even be harmful. These are especially dangerous for people who value truth and correctness, but are not politically savvy. (ex: Genetics research both today and in Soviet Russia)
4) The model is easy to misuse. (ex: p-values)
5) Consequences of misuse are severe - More bad news: even if a correct model is hard to misuse, it can still lead to bad outcomes! This happens when using the model wrongly is catastrophic. (ex: Immigration)