Ultraprocessed food is a product of an optimizer that cares about a proper subset of human values, and Goodharts-away the others.
When you're cooking food for your family from scratch, you get to decide what to optimize for. You can consider variables like satiety, nutrition, taste, and cost; and decide for yourself what function of these you want to maximize.
It would be a surprising coincidence if the function that home cooks maximize, turned out to be identical to the function that the processed food industry maximizes. The processed food industry doesn't have the same incentive structure as home cooks. It doesn't live in your house and take care of your kids; it lives in a competitive free market. What it can optimize for is constrained by market economics: it is capable of caring about human values like satiety, nutrition, and taste only insofar as these matter to selling more food.
For instance, home cooks hardly ever try to reduce satiety; but industry often does: "once you pop, you can't stop"; "less filling, tastes great" are ad slogans for snack and beverage products that represent something that the market values: you will eat and drink (and buy) more of this, specifically because it is less satiating.
But the correct satiety setting for maximizing sales is not the correct satiety setting for maximizing health.
Ultraprocessed food is a product of an optimizer that cares about a proper subset of human values, and Goodharts-away the others.
That's true in general, but I don't think that's quite the reason in the specific case of obesity. Because I'm pretty sure that if it was definitively known and common knowledge what exactly it was that was causing obesity, food manufacturers would trip over themselves to get rid of it and advertize their product as being free of the factor.
The optimizer does care about this—Atkins! Fat free! Organic! Sugar free! GMO free!
Instead, we have a subtly different problem, where some parts of the optimization have tighter loops than others, and this causes a temporary Goodhart effect in the meantime while we struggle to close certain loops.
Now I could be wrong about this specific case: maybe the obesogenic is directly addictive for example, or the cost to remove it would be too much for the average consumer to pay for. But I think it still illustrates an important principle for mild optimizers: you can't safely run too far ahead of your slowest loop.
There's not an ingredient that causes obesity. What causes obesity is a food culture driven by an optimizer that wants to put more food into bodies; that uses all manner of means, conscious and unconscious, to cause more food to go into bodies. And a major approach is avoiding satiety and forming habits that avoid satiety, because satiety is the signal that causes food to stop going into the body.
It's not that people don't respond to health claims printed on food products. It's that printing health claims on food products is entirely subservient to getting more food into bodies than otherwise would get into them.
Traditional human food culture is optimized to, among other things, get nutrition out of foodstuffs. Industrial food is optimized to sell more foodstuffs. These are not the same goals ... and as you point out, the sales optimizer runs a tighter loop than the human food culture optimizer. Sales runs on a quarterly cycle; human food culture runs on a generational cycle. The sales optimizer gets to control the (much slower) development of lifelong food preferences; and this means it gets to form habits that deliver more food into bodies.
This is a "temporary Goodhart effect" on the same scale as the temporary boom in lung cancer in the 20th century. And on the personal level, a lot of this cashes out as cultural patterns of relating to food the way a smoker relates to cigarettes; as a matter of craving and habit, of "just one more, even though I know it's bad for me," food as fun, food as rebellious fun specifically ("sinfully rich chocolate!"), food as non-satiating social activity, and so on.
The optimizer consistently reinforces these patterns, because doing so sells more food. If you don't get your consumers to form habits around your product, you sell less product, and you get bought out by the company that knows marketing better. You sell more corn chips if you actually try to get more corn chips into bodies ... which is to say, if you actually try to get people to do the thing that causes obesity.
Tobacco was introduced to Europeans in the 1600s, but lung cancer didn't really catch on until the 20th century because that was when industrial manufacture and marketing of cigarettes made it possible to get much more tobacco into bodies; to create not only habits but two-pack-a-day habits. Cigarette smoking was consciously promoted into a central role in culture, from the workplace (the smoke break) to the bedroom (the post-sex smoke). The amount of tobacco consumed per capita increased dramatically, and that's when the public health effects became overwhelmingly clear. (And smoking peaked around 1970 and has been in decline ever since. There's the temporary boom.)
Industrial food has done the same. The food sales optimizer wants to get more food into bodies, just as the cigarette sales optimizer wanted to get more tobacco into bodies. And the consequence of putting more food into bodies is obesity.
1As far as I know, they could generally all afford as much food as they wanted.
Unless your great grandparents were rich, this seems unlikely. People massively underestimate how much worse and more expensive food was in the recent past. In the 1950's, people spent a lot of their time thinking about how to get cheaper food, and eating food we would consider to be not particularly good (pretty much everything canned).
Similarly, the best food you mom knows how to make is probably not representative of what your family was eating regularly in your grandparents' generation.
Going to the gym for 20 minutes every other weekend burns more calories than a 9 hour shift cooking food every day.
I think this is incorrect by a huge margin. Working out actually doesn't burn very many additional calories at all.
Hmm maybe you are right about my great-grandparents food availability, I should look into it more.
The second quote was meant to be a joke.
1One thing to consider is food expenditure over time:
People spent twice as much on food in 1950, despite eating out half as much and mostly cooking foods we'd consider extremely cheap today.
What is it about processed food that's so bad for you? The slicing? The smashing? The pressing? The freezing? The drying? The boiling? The canning? The conveyor belts? Is it simply filled with malice by the people who create it?
Slicing, smashing, pressing, etc. increase the glycemic index. Processing typically includes adding salt and sugar.
Imagine that your kids told you: "If you want me to eat the broccoli, could you at least put some sugar on it to make it taste better?". You would probably say: "Are you crazy? Sugar on broccoli? Stop saying nonsense, and eat what's on your plate."
But if a CEO of a food producing company heard that, his first reaction would be: "Hmmm, I wonder if there is a right amount of sugar we could add to the broccoli, enough that the customers would go 'hey, I don't know why, but this broccoli just somehow tastes better than the broccoli from other brands', but not enough that they would go 'ewww, broccoli with sugar, wtf'."
Even worse, first only one company does it, but if it succeeds, other companies start doing the same thing. Soon a generation grows up that considers that taste of broccoli to be normal. Then one company tries adding a little more sugar to get an advantage over the competitors, and the whole cycle happens again. Five generations later, all food are disgustingly sweet (for a foreigner, but normal for locals), and people wonder what makes them fat.
No new evidence is presented.
What is it about processed food that's so bad for you? The slicing? The smashing? The pressing? The freezing? The drying? The boiling? The canning? The conveyor belts? Is it simply filled with malice by the people who create it?
How come none of my 8 great grandparents were fat but 3 of my 4 grandparents were? They all sat at desks or worked in factories or took care of the home. They didn't have terribly different lifestyles. As far as I know, they could generally all afford as much food as they wanted. (By the way, my 4th grandparent has lived her whole adult life on unincorporated islands.)
I would choose my mom's soup over chikfila soup every time; I would choose my mom's burgers over McDonald's every time. I wouldn't say that fast food or frozen food or canned food or prepared meals are better or more palatable than homemade food at all. And my grandma was a better chef than my mom and my great grandma was a better chef than my grandma. Food is getting less palatable, not more.
What good explanations are left?
We all know that higher "socioeconomic status" magically makes you healthier and skinnier. Going to the gym for 20 minutes every other weekend burns more calories than a 9 hour shift moving boxes every day. But perhaps there is a second factor. If your spouse doesn't work a job than they'll have more time to cook. I posit some kind of magic enhancement mechanism is going on here, where if your parents or spouse cook breakfast and dinner then you get a boost to the inexplicable wealth-health magic.