Sounds like same way we had a dumb questions post we need somewhere explicitly for posting dumb potential solutions that will totally never work, or something, maybe?
Given he is going to be doing this at literal OpenAI, how confident are we that this is on net a good idea? I'm especially interested in Christiano's opinion here, since he was Aaronson's student and he also was at but left OpenAI.
Appreciation for sharing the reasoning. Disagreement with the reasoning.
eeegnu is saying they go where their ideas take them and expressing ethical qualms, which both seem like excellent reasons to want someone considering AI safety work rather than reasons to drive them away from AI safety work. Their decision to continue doing AI safety work seems likely to be correlated with whether they could be productive by doing additional AI safety work - if their ideas take them elsewhere it is unlikely anything would have come of them staying.
This is especially true if one subscribes to the theory that we are worried about sign mistakes rather than 'wasting' funding - if we are funding unproven individuals in AI Safety and think that is good, then this is unusually 'safe' in the sense of it being more non-negative.
So to the extent that I was running the LTFF, I would have said yes.
Anyone have good real life examples of 2nd Level Goodhart to throw out there? (e.g. where you have T as a measure of U which is supposed to be a measure of V, but you can't measure U directly either, so you end up optimizing for T). Can be either 'it works out mostly OK' or where it works out totally not OK.
Maybe. I am torn between 'this does seem like an important thing no one has noticed or at least no one has pointed out explicitly, that I understand and could explain to them' and 'but that's the whole post, the rest is obvious, no?'
Fair, but my experience says this is true even for Area Man, although Area Man will have a harder time meeting the bar.
My guess is about a thousand, mostly because if you have a 'strange general election' where you have two Rs or two Ds then Biden and Trump both are highly polarizing without being highly unifying, and Biden has the economy as an albatross around his neck right now, so I'd expect - if the other candidate was reasonable - them to lose the other party and still lose some of their own party, which lets otherwise much worse candidates have a chance. But we're still talking about e.g. 900 politicians and 100 business people and maybe a handful of very famous celebrities. and that's it.
Technical note: Most of the time that Biden or Trump loses it is because they literally died during the month of campaigning, or had a huge health issue where they are obviously no longer able to serve. It's not that unlikely.
A bunch of it was novel to me, and it represents thinking that is clearly their own, but I am not familiar with the literature in question.
Yes. The politician wins. Focus groups and polls are clear on this, they won't vote for random people. Trump would also beat them easily.
(Obviously some probability you get someone effectively not random from a random draw, but it's very low.)
Good note on the title. Hopefully the first paragraph cleared that up, but I'll watch for that in the future. Likely too late at this point to fix.