Highlights

OpenAI Charter: In their words, this is "a charter that describes the principles we use to execute on OpenAI’s mission".

My opinion: I'm very excited by this charter, it's a good sign suggesting that we can get the important actors to cooperate in building aligned AI, and in particular to avoid a competitive race. Key quote: "if a value-aligned, safety-conscious project comes close to building AGI before we do, we commit to stop competing with and start assisting this project".

Lessons Learned Reproducing a Deep Reinforcement Learning Paper (Matthew Rahtz): It's exactly what the title says. There were a lot of points that I can't easily summarize, but some highlights:

My opinion: If you do deep RL research regularly, you probably won't get too much out of it (though you might still get some handy tips on things you can do with Tensorflow), but I think everyone else should read it to get a more concrete sense of what deep RL research actually looks like and to be able to communicate more effectively with deep RL researchers.

Read more: Deep Reinforcement Learning Doesn’t Work Yet

A voting theory primer for rationalists and 5 voting pathologies: lesser names of Moloch (Jameson Quinn): Voting theory, or social choice theory, studies voting methods, which take a set of preferences over outcomes from voters, and decides which outcome should occur. The field is littered with impossibility results and difficult problems that are hard to resolve, but there are voting methods that do reasonably well in practice. The second post elaborates more on the 5 problems that are hard to resolve.

My opinion: A major challenge for an AI would be to figure out how to aggregate preferences across humans. This is not necessarily a problem that we have to solve immediately -- it's possible that we build an AI that mimics what we do initially and then develops its own theory of voting. However, if you are doing any kind of reward learning (such as inverse reinforcement learning), you will have to confront this problem head on. This article points out a lot of problems that we would have to be aware of in this case. The solutions seem less likely to transfer, because they are optimized for a different scenario (such as presidential elections).

Read more: Will MacAskill's PhD thesis, Normative Uncertainty, considers how to comb...