the listening experience would have been better with another narrator
Perhaps, but I'm liking the narration so far. I find it about as good as your narration of your book, and even perhaps a bit better.
At the beginning of Chapter 2, Mollick misattributes the idea of a "paperclip maximizer" to Bostrom. Yudkoswky is the actual originator of the idea. Source: https://www.lesswrong.com/tag/squiggle-maximizer-formerly-paperclip-maximizer
Very helpful reply, thank you!
(My salary has always been among the lowest in the organization, mostly as a costly signal to employees and donors that I am serious about doing this for impact reasons)
I appreciate that!
I have completely forfeited my salary, and donated ~$300k to Lightcone at the end of last year myself to keep us afloat
If you had known you were going to do this, couldn't you have instead reduced your salary by ~60k/year for your first 5 years at Lightcone and avoided paying a large sum in income taxes to the government?
(I'm assuming that your after-tax salary from Lightcone from your first 5-6 years at Lightcone totaled more than ~$300k, and that you paid ~$50k-100k in income taxes on that marginal ~$350k-$400k of pre-tax salary from Lightcone.)
I'm curious if the answer is "roughly, yes" in which case it just seems unfortunately sad that that much money had to be unnecessarily wasted on income taxes.
I originally missed that the "Expected Income" of $2.55M from the budget means "Expected Income of Lighthaven" and consequently had the same misconception as Joel that donations mostly go towards subsidizing Lighthaven rather than almost entirely toward supporting the website in expectation.
Something I noticed:
"Probability that most humans die because of an AI takeover: 11%" should actually read as "Probability that most humans die [within 10 years of building powerful AI] because of an AI takeover: 11%" since it is defined as a sub-set of the 20% of scenarios in which "most humans die within 10 years of building powerful AI".
This means that there is a scenario with unspecified probability taking up some of the remaining 11% of the 22% of AI takeover scenarios that corresponds to the "Probability that most humans die because of an AI takeover more than 10 years after building powerful AI".
In other words, Paul's P(most humans die because of an AI takeover | AI takeover) is not 11%/22%=50%, as a quick reading of his post or a quick look at my visualization seems to imply, but is actually undefined, and is actually >11%/22% = >50%.
For example, perhaps Paul thinks that there is a 3% chance that there is an AI takeover that causes most humans to die more than 10 years after powerful AI is developed. In this case, Paul's P(most humans die because of an AI takeover | AI takeover) would be equal to (11%+3%)/22%=64%.
I don't know if Paul himself noticed this. But worth flagging this when revising these estimates later, or meta-updating on them.
I have time-space synesthesia, so I actually picture some times as being literally farther away than others.
I visualize the months of the year in a disc slanted away from me, kind of like a clock with New Years being at 6pm, and visualize years on a number line.
"[A] boy born on a day that I'll tell you in 5 minutes" is ambiguous. There are two possible meanings, yielding different answers.
If "a boy born on a day that I'll tell you in 5 minutes" means "a boy, and I'll tell you the name of a boy I have in 5 minutes" then the answer is 1/3 as Liron says.
However, if "a boy born on a day that I'll tell you in 5 minutes" means "a boy born on a particular singular day that I just wrote down on this piece of paper and will show you in 5 minutes", then this is equivalent to saying "a boy born on a Tuesday" and the answer is 13/27.
The reason why the second meaning is equivalent to "a boy born on a Tuesday" is because it's a statement that at least one of the children is a particular kind of boy that only 1/7th of boys are, just like how "a boy born on a Tuesday" is a statement that at least one of the children is a particular kind of boy that only 1/7th of boys are. (Conversely, for the first interpretation: "a boy born on a day that I'll tell you in 5 minutes" is a statement that at least one of the children is a a boy, period.)
Another way to notice the difference if it's still not clear:
When told "I have two children, at least one of whom is [a boy born on a particular singular day that I just wrote down on this piece of paper and will show you in 5 minutes]", you assign a 1/7th credence to the paper showing Sunday, 1/7th to Monday, 1/7th to Tuesday, etc.
Then, conditional on the paper showing Tuesday, you know that the parent just told you "I have two children, at least one of whom is [a boy born on [Tuesday and I will show you the paper showing Tuesday in 5 minutes]]", which is equivalent to the parent saying "I have two children, at least one of whom is a boy born on Tuesday".
So you then have a 1/7th credence that the paper shows Tuesday, and if it's Tuesday, your credence that both children are boys is 13/27. So your overall credence, reflecting your uncertainty about what day the paper shows is (13/27)*(1/7)+(13/27)*(1/7)+(13/27)*(1/7)+(13/27)*(1/7)+(13/27)*(1/7)+(13/27)*(1/7)+(13/27)*(1/7)=13/27.