I'm Jérémy Perret. Based in France. PhD in AI (NLP). AI Safety & EA meetup organizer. Information sponge. Mostly lurking since 2014. Seeking more experience, and eventually a position, in AI safety/governance.
Extremely annoyed by the lack of an explorable framework for AI risk/benefits. Working on that.
There is a collection of posts that are best at making this point. This one is part of it, and most of the others don't exist.
I also have the (Pareto) Best in the World concept in my back pocket. I used it to reassure me that I can shine in some way. Somehow, believing that I'm The Best at whatever I'm doing is extremely gratifying and drive me to do more things. I would prefer this belief to be true, and drive is useful. So I keep trying and combining my skills in novel way.
This post helps by providing clear examples, to communicate the strategy of seeking/filling gaps and leveraging teamwork.
Expecting people beyond myself to have this motivational quirk, I have used a variant of this post's arguments as pep talk, with some success.
Seconding Viliam's comment, this post clarifies an important assumption of Ricardo's Law quite well. I have been using a version of this formulation to restate Eliezer's original point in a smoother way, with positive feedback.
I took the survey! Thank you for running it once again.
(This is the comment I propose as Schelling point to collect replies saying you took the survey, to make the comment section tidier)
Let's see if your post has successfully overcome my mental filters (at the very least, I clicked). Here's my reformulation of your claims, as if I had to explain them to someone else.
Now that I've written the points above, I study again the "what if" part at the end and say, "oh, so the idea is that human language may not be the best way to transmit knowledge because what gets your attention often isn't what lets you learn easily, cool, then what"
Then... you claim that there might be a Better Language to cut through these issues. That would be extremely impressive. But then I scroll back up and I see the titles of the following posts. I'm afraid that you will only describe issues with human communication without suggesting techniques to overcome them (at least in specific contexts).
For instance, you gave an example comparison in impression (asteroid vs. climate change). Could you provide a comparison for relevance? Something that, by your lights, gets processed easily?
they abandoned simple metrics in favour of analyses in which qualitative factors play a large role, because all the metrics they evaluated failed to have good properties
Do you have more specific statements from GiveWell for this shift? I have not been able to find a clear enough argument for your claim from their website, nor from research on the EA Forum.
Also, your view on well-behaved utility functions may vary. You need to get an approximation of ideal utilitarianism, with a nice ordering of world-states by total happiness/suffering (depending on flavor) and how to get there. I think we can coordinate on some good enough approximations to be able to give. Is that well-behaved enough, or are you pointing at something stronger here?
I am now curious about the omission of French, hoping that's because you already have competent people for it, maybe the aforementioned kind souls?
Related recent post: Intelligence Is Not Magic, But Your Threshold For "Magic" Is Pretty Low (similar point, focused on human short-timeframe feats rather than technological achievements).
Format note: your list is missing a number 3.
[Fanfiction, continued from yours, wasn't sure if I got the message, please correct me if I went the wrong way]
The master traveler eventually returned to the temple, richer from having successfully led the caravan through and back. He approached the cartographer again, and gave them a small notebook with a nod.
The cartographer's student was confused. "Teacher, have you heard of the other caravan having gone recently through the northern mountains? It seems they didn't make it. Why is that so, when you gave them the same map?"
"I could say," answered the cartographer, "that the earlier one had been unlucky. Wild animals? Bad weather? But I've seen enough travelers to understand the patterns. The earlier guide's eyes followed the general shape of the map; how it was drawn, not what was drawn, because they were not familiar with the territory. Hence, they could not follow the map, and probably lost themselves."
"But teacher," asked the student, even more confused, "what is the use of a map, if not to guide a traveler through territories unknown to them? How does a map provide information to those who already possess it?"
"My dear, most maps are scaffolds, not recipes. Masters use it for planning, for the details they haven't bothered remembering, for emergency detours. The default path, they don't need a map for. The rest, they shall check by experiment. Hence," the cartographer waves the traveler's notebook, "the observations that will add to my sources. Our next map shall be better for them."
"Teacher, I beg your patience, and yet, if the default path had been misrepresented, surely the master would have noticed. Which means the novice had the correct map of it? How could they be led astray by truthful directions?"
The cartographer looked in the distance and sighed. "So rarely are those directions actually followed, and verified often enough. A novice may focus on the next shelter they want to reach eight hours thence, and take a wrong turn an hour later. They will look at the mountains further away, tell themselves they're indeed in the north, as the map says, but that observation fits many paths, right and wrong. Whereas, if they should have crossed a river and didn't, even if they notice later, they might not remember when they went wrong, and will look at their immediate surroundings to find how the map could tell them they are where they are supposed to be. A master trusts the territory above the map, and the map above their own hasty judgment."