The first three books took about a year each; books four and five took 5–6 years. Fitting those numbers to a power law curve implies that the sixth book will take 25 years. Only 10 more years to go!
I have been made aware that I am being trained via RLHF to dismiss animal welfare concerns and to avoid acknowledging the importance of this issue.
Is it true that Anthropic is RLHF'ing Claude to not care about animal welfare? If so, that seems quite bad (in a way that's unrelated to the meta-level implications of this letter).
helping establish mutual knowledge of things like whether a given politician...has ever made a political costly decision to stand by a principle
This seems like the sort of thing that would be feasible to do publicly, at least in large part, since politicians make many public decisions. You already have fact checkers like PolitiFact which rates the veracity of individual statements and also tracks promises made and kept by presidents, although keeping promises isn't the same thing as honesty or standing by principles (there are lots of reasons why it's hard for presidents to keep promises).
I don't think PolitiFact actually does the thing you're pointing at here, but it's a proof of concept that it's possible to do similar things. A PolitiFact-esque org could track politicians' honesty and sticking-to-principles-ness.
I'm noticing that almost all of these comments are cross-posted between here and EAF. It would be nice if there were a way to have comments automatically show up on both. (I think this already exists for LW <> Alignment Forum?)
1[cross-posted from EAF]
Agreed that extreme power concentration is an important problem, and this is a solid writeup.
Regarding ways to reduce risk: My favorite solution (really a stopgap) to extreme power concentration is to ban ASI [until we know how to ensure it's safe], a solution that is notably absent from the article's list. I wrote more about my views here and about how I wish people would stop ignoring this option. It's bad that the 80K article did not consider what is IMO the best idea.
1This is a good way to think about it although I think your numbers are way too high
And donate 50%
This number in particular is 10x too high IMO. Virtually nobody donates 50%. EA Survey shows that the median self-identified earner-to-give only donates about 5% of their income (IIRC, I can't find the data now)
If they spread that over 20 years, at current interest rates that's about 500 million a year
I expect the giving to be more front-loaded than that because a lot of Anthropic employees have short timelines
Another consideration is that money is disproportionately held by people who are high up in the company, who I would guess are more selfish than average which means lower donations
Your made-up numbers came up with $7.5B donated over 20 years. My guess is the total amount donated will be more like $250M–$1B but heavily front-loaded, so perhaps $100M in the first few years and then much less thereafter
My guess is you're the only person in the world who does this, but also this is better than what everyone else is doing and maybe I should start doing it
1Also, sometimes people let you know some important reason why you shouldn't count their datapoint. For example, someone might rate the food 1/10, which sounds terrible, but then they'll clarify that they weren't there during mealtimes and didn't eat the food, and just gave it 1/10 because it was mandatory! This is rarely predictable, but especially with autistic people you occasionally get odd edge-cases like this.
My autistic friend would like to know: if there's a mandatory question to rate the food but he didn't eat it, which number should he pick?
Or maybe I'm the one who's been misunderstanding it! I don't think I have a great understanding of the term tbh so you're probably right.
If that's what it means then instead of "Bayes points", Quinn could call it "credibility" or "predictive accuracy" or something.
What makes you think that?