Another reason why not...
Aurini: there is one branch of government which does a great job of [the abyssal scenario] - the military.
Millenium Challenge. (A defence of it is here for sake of balance).
I'd expect lots of Advance Abyssal plans by government, especially war gaming as Eliezer suggested, to end up something like that. As a hypothetical future it falls more prone to signalling and various biases.
Most picked (a).Ack, brain fell out! I meant they picked (b).
Otus, I'd pick your first option. I don't think the safety of (2%, 2%) is worth the 0.5% expected value loss. If the risks were huge, say (-20%, 30%) and (2%, 2%) I'd probably play safe.
Fascinating comment log, especially given how smart the players were.
I can't find it online, but back in the "OMG the Japanese are eating us for breakfast" days, a study asked ordinary Americans: would you rather...
a) Our economy grow 2% next year and the Japanese 4%. b) Both economies grow 1%.Most picked (a).
I'll be downright indignant if Reality says "So what?" and has the superintelligence make slower progress than human engineers instead.
Maybe once it's secure from being overtaken by rivals it slows down to really nail down the safety aspect, and make sure the sky isn't tiled with representations of satisfied superintelligent utility functions...
To put my point another way...
Say your experts, above, predict instance frequencies in populations. Huge populations, so you can only test expert accuracy by checking subsets of those populations.
So, you write an algorithm to choose which individuals to sample from populations when testing expert's beliefs. Aren't there situations where the best algorithm, the one least likely to bias for any particular expert algorithm, would contain some random element?
I fear randomness is occasionally necessary just to fairly check the accuracy of beliefs about the world and overcome bias.
Gregory: Either way, it implies the presence of intelligence. Scott: If it wasn't, it occurred "naturally" - that is was constructed through basic physical phenomena.
Probably, but not necessarily. We've already met on our own planet a non-intelligent optimization process that's built gears to "whir rapidly, perfectly meshed", and electricity generators and conductors for both data transmission and delivery of large currents, and surfaces that gleam in bright colours, and even built things using metals.
Can I guess that the machine's makers were intelligent, without guessing their motivations?
We can guess optimization, but I'd avoid unconsciously assuming it wasn't built by an unintelligent optimizer, such as some weird alien evolutionary process or non-intelligent creature/hive, without more extra-terrestrial data.
We've a Saturday 7.30pm reservation for 21 people at Il Fornaio, 302 S Market St, San Jose, in the Sainte Claire Hotel.
They have a wide variety including the requested vegetarian options & pizza from an oak wood burning oven, mostly good reviews, casual dress code, the Wine Spectator Award of Excellence 2007, and are just down the road from Montgomery Theater.
7.30 should give people time to finish chatting at the theater or freshen up at their hotels, but if you want to come a bit earlier they say they'll probably be able to seat you. If lots of people think 7.30 is too late I could change the time or suggest a cafe for early birds to grab a coffee in before dinner.
Please RSVP whether or not you'll be coming, either in a comment or email me at cursor_loop 4t yahoo p0int com. That way if new people want to come as they can take unclaimed places, and I can let the restaurant know if the number changes. If you think you'll be late, please let me know that too so we can ask the restaurant to keep your seat.
Also, as my cellphone won't work San Jose, please could someone volunteer their number for unexpectedly late or lost people to ring?
Hope to see you all on Saturday!
Update: 2 Vegetarians (so vegetarian options will be checked for), one more preference for pizza, and a preference against Mexican.
Pizza is winning! Anyone who doesn't like it, let him speak now or forever hold his peace of pizza.
This is becoming quite an interesting diffusion of responsibility experiment.
PJ Eby, if you've not seen it, you may enjoy this.