I think a significant amount of the probability mass within P(no AI takeover) is in various AI fizzle worlds. In those worlds, anyone outside AI safety who is working on making the world better, is working to increase the flourishing associated with those worlds.
1You can’t require people to “be safe” or “be kind” or “avoid suffering” – or, at least, you can’t do that using any of the legal and policy tools common in English-speaking countries.
This is not true, the entire area of Tort law (especially negligence law) is highly developed in English speaking countries, and is a series of elaborations on "take reasonable care to avoid harm to others."
One thing you might be missing, and the reason that widely-viewed events are so valuable as ad space, is that one function of ads can be to create common knowledge among the audience of how other people might see the product.
For example, Corona beer has a beach vibe. They don't just want you to know that it has a beach vibe, they want you to know that other people know that it has a beach vibe. That way, if you are going to a party and want to bring something that has a beach vibe, you'll reach for Corona because you want to be seen by others as bringing the beach vibe. This is why ads on widely-viewed broadcasts like sports games disproportionately (tho not exclusively) focus on products that are consumed socially or in public rather than privately.
1I often think about iconic comic book characters as essentially today's version of these kind of mythical stories, which cut across people's disagreements about religion. The story of Spider Man has been retold many times but generally contains a classic arc:
- Peter finds himself unexpectedly with a lot of power all of a sudden.
- Peter has an opportunity to use that power to do the right thing at effectively low cost to himself, and chooses not to for selfishness/laziness reasons.
- His choice not to do right has unforeseeable negative consequences both for society and for him personally.
- He then dedicates himself to saving people/fighting crime/etc.
- In many versions of the story, he pays a lot of personal costs for the time and energy he spends helping others, but he still does it.
- In many versions, at some point he gets burned out or panicked that it feels like his power to do good becomes an obligation he can't escape, and he feels distraught by all the people he cannot help.
- In many versions, he eventually (through some dramatic event) comes to terms with his inability to save everyone, and learns that what he can accomplish is still valuable and good, even if he can't save everyone.
I think its essentially an aspirational model of what it means to be a good person living a balanced life as you come of age and gain authority, power, and responsibility over others. And I do think that people absorb these truths from the story.
One other thing has to do with the procedural posture. On a motion to dismiss, Courts are not generally supposed to evaluate facts at all. A motion to dismiss is essentially an argument by the defense that, even if all those accusations are true, they don't amount to a legal claim.
By contrast, a preliminary injunction does look at facts. So one reason for different treatment of the Encode brief vs. the ex-employees brief is that they were submitted in response to different motions.
One of the most common forms of Whataboutism is of the form "You criticize X, but other people vaguely politically aligned with you failed to criticize Y." (assuming for argument that X and Y are different but similar wrongs)
The problem with that is that the only possible sincere answers are necessarily unsatisfying, and it's hard to gauge their sincerity. Here's what I see as the basic possibilities.
- Y and X are equally bad, my allies are wrong about this [but what are you gonna do about it?]
- Y is bad but X is genuinely worse because of .... (can sound like a post hoc justification)
- Y and X are equally bad, but I still support my side because the badness of Y is outweighed by the goodness of A, B, etc.
- Y is actually not bad because of ....
- You are right, Y is terrible, I abandon my allies [and join ... what? If Y is disqualifying X surely is too...]
The PCC is a lot more valid when its actually the same person taking inconsistent positions on X and Y. Otherwise your actual interlocutor might not be inconsistent at all but has no plausible way of demonstrating that.
The problem with this argument is that it ignores a unique feature of AIs - their copiability. It takes ~20 years and O($300k) to spin up a new human worker. It takes ~20 minutes to spin up a new AI worker.
So in the long run, for a human to economically do a task, they have to not just have some comparative advantage but have a comparative advantage that's large enough to cover the massive cost differential in "producing" a new one.
This actually analogizes more to engines. I would argue that a big factor in the near-total replacement of horses by engines is not so much that engines are exactly 100x better than horses at everything, but that engines can be mass-produced. In fact I think the claim that engines are exactly equally better than horses at every horse-task is obviously false if you think about it for two minutes. But any time there's a niche where engines are even slightly better than horses, we can just increase production of engines more quickly and cheaply than we can increase production of horses.
These economic concepts such as comparative advantage tend to assume, for ease of analysis, a fixed quantity of workers. When you are talking about human workers in the short term, that is a reasonable simplifying assumption. But it leads you astray when you try to use these concepts to think about AIs (or engines).
This idea kind of rhymes with gain-of-function research in a way that makes me uncomfortable. "Let's intentionally create harmful things, but its OK because we are creating harmful things for the purpose of preventing the harm that would be caused by those things."
I'm not sure if I can formalize this into a logically-tight case against doing it, but it seems conceptually similar to X, and X is bad.
For a much less gameable and more fun version of Calibration Trivia, use the rules that the bar I play trivia at uses.
The game is broken up into rounds. Within a round, players/teams have 2, 4, 6, 8, and 10 points that they can assign to their answer as they submit it. They can use each only once. Here is the sequence of events:
You can adapt this basic structure to test calibration more or less. You could, e.g., read all 5 questions and have folks assign the point values to all 5 answers before turning any of them in, or require assigning point values to the categories before hearing any of the questions. You could make the points a wager (allowing negative scores) - but allow people to bet zero some number of times. You could increase/decrease/change the point values per round.
But this basic structure allows you to reward good calibration without rewarding stupid calibration. Knowing some answers in this game is necessary but not sufficient to win.