GiveWell interview with major SIAI donor Jaan Tallinn

byjsalvatier8y19th Jul 20118 comments


GiveWell recently release notes from their interview with Jaan Tallinn, Skype co-founder and a major SIAI donor, about SIAI (link). Holden Karnofsky says 

[M]y key high-level takeaways are that

  1. I appreciated Jaan's thoughtfulness and willingness to engage in depth. It was an interesting exchange.
  2. I continue to disagree with the way that SIAI is thinking about the "Friendliness" problem. 
  3. It seems to me that all the ways in which Jaan and I disagree on this topic have more to do with philosophy (how to quantify uncertainty; how to deal with conjunctions; how to act in consideration of low probabilities) and with social science-type intuitions (how would people likely use a particular sort of AI) than with computer science or programming (what properties has software usually had historically; which of these properties become incoherent/hard to imagine when applied to AGI)