We spend enormous energy protecting bank statements at some level, treating them as surveillance risks. Yet people voluntarily share with AI systems something far more revealing: how they actually think, what they believe, what they are uncertain about, etc. A bank statement can tell you someone eats at the same place twice a week. But a long AI conversation tells you how their thinking drifts, what they are afraid of, what kinds of arguments move them, and thoughts they would never say out loud. I think the most sensitive data is no longer behavioural.
My take is that, current models are trained on enormous amounts of data and learn to produce what most people find good or useful. But when you ask them to create something genuinely new, they don't really think, rather they recombine what they have seen most often, or at least this seems to me like that. The result rarely feels truly unique. I wonder if this is also why they tend toward mediocrity. If you train on everything, you optimise for the average. The average is not where the interesting work is.
Having studied law, computer science, and art histor...
The findings are interesting. It suggests models generalise better when they understand reasons rather than just patterns. Which leads to question if explaining why a rule exists already helps, could the next step be abstracting across rules to find the underlying principles that generate them? Rather than teaching models a better map of rules, teaching them what produces good rules in the first place.