Recently someone either suggested to me (or maybe told me they or someone where going to do this?) that we should train AI on legal texts, to teach it human values. Ignoring the technical problem of how to do this, I'm pretty sure legal text are not the right training data. But at the time, I could not clearly put into words why. Todays SMBC explains this for me:
Law is not a good representation or explanation of most of what we care about, because it's not trying to be. Law is mainly focused on the contentious edge cases.
Training an AI on trolly problems and other ethical dilemmas is even worse, for the same reason.
I don't read much sensationalist tabloid, but my impression is that the things that get a lot of attention in the press, is things people can reasonable take either side of.
Scott Alexander writes about how everyone agrees that factory framing is terrible, but exactly because this overwhelming agreement, it get's no attention. Which is why PETA does outrageous things to get attention.
The Toxoplasma Of Rage | Slate Star Codex
There need to be two sides to an issue, or else no-one gets ingroup loyalty points for taking one side or the other.
3CstineSublime
A cursory glance suggests that it is not the case, take a top story headline on the Australian Daily Mail over the last 7 days: "Miranda, Sydney: Urgent search is launched for missing Bailey Wolf, aged two, who vanished yesterday" it is not reasonable for someone to hope that a two year old who has vanished not be found. This is exactly the kind of thing you're suggesting AI should be trained on, because of how uniform responses are to this headline. Keep in mind this is one of the most viewed stories, and literally top of the list I found.
I've read Scott's article, but are you trying to understand what get's attention or what is the nexus or commonly agreed upon moral principles of a society?