How are you going to address the perceived and actual lack of rigor associated with SIAI?

A clarifying question. By 'rigor', do you mean the kind of rigor that is required to publish in journals like Risk Analysis or Minds and Machines, or do you mean something else by 'rigor'?

A clarifying question. By 'rigor', do you mean the kind of rigor that is required to publish in journals like Risk Analysis or Minds and Machines, or do you mean something else by 'rigor'?

I mean the kind of precise, mathematical analysis that would be required to publish at conferences like NIPS or in the Journal of Philosophical Logic. This entails development of technical results that are sufficiently clear and modular that other researchers can use them in their own work. In 15 years, I want to see a textbook on the mathematics of FAI that I can put... (read more)

3XiXiDu9yI can't speak for the original questioner, but take for example the latest post [] by Holden Karnofsky from GiveWell. I would like to see a response by the SIAI that applies the same amount of mathematical rigor to show that it actually is the rational choice from the point of view of charitable giving. A potential donor might currently get the impression that the SIAI has written a lot of rather colloquial posts on rationality than rigorous papers on the nature of AGI, not to mention friendly AI. In contrast, GiveWell appears to [] concentrate on their main objective, the evaluation of charities. In doing so they are being strictly technical, an appraoch that introduces a high degree of focus by tabooing colloquial language and thereby reducing ambiguity, while allowing others to review their work. Some of the currently available papers [] might, in a less favorably academic context, be viewed as some amount of handwaving mixed with speculations.

Q&A with new Executive Director of Singularity Institute

by lukeprog 1 min read7th Nov 2011182 comments


Today I was appointed the new Executive Director of Singularity Institute.

Because I care about transparency, one of my first projects as an intern was to begin work on the organization's first Strategic Plan. I researched how to write a strategic plan, tracked down the strategic plans of similar organizations, and met with each staff member, progressively iterating the document until it was something everyone could get behind.

I quickly learned why there isn't more of this kind of thing: transparency is a lot of work! 100+ hours of work later, plus dozens of hours from others, and the strategic plan was finally finished and ratified by the board. It doesn't accomplish much by itself, but it's one important stepping stone in building an organization that is more productive, more trusted, and more likely to help solve the world's biggest problems.

I spent two months as a researcher, and was then appointed Executive Director.

In further pursuit of transparency, I'd like to answer (on video) submitted questions from the Less Wrong community just as Eliezer did two years ago.


The Rules

1) One question per comment (to allow voting to carry more information about people's preferences).

2) Try to be as clear and concise as possible. If your question can't be condensed into one paragraph, you should probably ask in a separate post. Make sure you have an actual question somewhere in there (you can bold it to make it easier to scan).

3) I will generally answer the top-voted questions, but will skip some of them. I will tend to select questions about Singularity Institute as an organization, not about the technical details of some bit of research. You can read some of the details of the Friendly AI research program in my interview with Michael Anissimov.

4) If you reference certain things that are online in your question, provide a link.

5) This thread will be open to questions and votes for 7 days, at which time I will decide which questions to begin recording video responses for.


I might respond to certain questions within the comments thread and not on video; for example, when there is a one-word answer.