Some additions about how the initial system is going to work:
Non-member contributions (comments and links) are going to become publicly visible when they have received 2 likes (from members---only members can like things).
However, members will be able to reply to a contribution as soon as it has received 1 like. This means that if you think someone's made a useful contribution, and you want to reply to them about it, you don't have to wait for a second person to Like it before you can write that reply. (It won't be publicly visible until the contribution has two Likes, though.)
Summary: We're planning to publicize and open up the forum very soon, and so it's a good time to discuss what we would like this forum to achieve, how we plan for moderation to work, and what discussions are on-topic.
Currently, this forum is read-only for everyone except for a few veterans of the mailing list it replaces. In a few days, we're planning to open up posting (in a tiered way, similar in spirit to the tiered privileges of MathOverflow), and the comments and Likes of the full members will play a material role in moderating the community. So it's a good time for those of us who are already here to discuss our goals for the forum, so that we stand a better chance of coordinating.
Right now, the forum has members who are allowed to submit posts and comments, and to Like other members' posts and comments; non-members can only read the contents of the forum, and membership is by invitation from the admins. What we're planning to do is allow non-members who sign up with their Facebook accounts to post links with titles, the way one can on Hacker News. (The Facebook login requirement reduces the amount of spam; clearly we'll want to add other ways of registering when we have the development time.)
These links from non-members will be visible only to members (and to the non-member who posted them) unless they've received some number of Likes from members, at which point they'll be publicly visible. Non-members who have posted a few links to good content that they wrote themselves will get membership invites. (We hope to also add the ability for members to recommend others for membership.)
The goal of all of this is to keep the discussion happening at a high level! One way of expressing the ideal of the forum is as a transparent box where high-quality conversation happens, and which has a mechanism for carefully letting new contributors in. That mechanism depends on the members, as well as the admins, maintaining an idea of what a good and relevant contribution looks like. (And there's really no way around doing some serious moderation if we want to create a useful place on the Internet. We aim for the standard set by Math Overflow and Hacker News.)
Now that I've explained the context, here is our current draft of the "how to contribute" page:
This is a publicly visible discussion forum for foundational mathematical research in artificial intelligence. The goal of this forum is to move toward a more formal and general understanding of "robust and beneficial" AI systems, as discussed in the Future of Life Institute's research priorities letter and the Machine Intelligence Research Institute's technical agenda.
Like Math Overflow, the Intelligent Agent Foundations Forum has a tiered system for becoming a full contributor. If you make an account with a Facebook login, you can post a link to an off-site contribution, e.g., on Medium or on a personal blog. These links will be visible to full forum contributors. If your link acquires enough Likes from full contributors, it will get promoted to visibility by all site visitors.
If you link to some good original content that you have written, the administrators will give you permissions to make posts and comments, and to Like others' contributions. The details of this system are still being worked out, and will change as we get a larger community of users.
So, what kinds of contributions are on-topic here? Here's one suggestion:
Definitely on-topic: the topics in MIRI's research agenda (decision theory, corrigibility, value learning, logical uncertainty, Vingean reflection, naturalized induction)
Generally on-topic: mathematical groundwork for self-modifying agents, abstract properties of goal systems, tractable theoretical or computational models of MIRI research topics
Not generally on-topic unless specifically connected to a relevant topic: recent advances in artificial intelligence and machine learning, existential risks, effective altruism, human rationality, mathematical logic, philosophy of mind
Please try not to encourage off-topic discussions; we'll have a button for reporting them to admins. (And yes, meta discussions are usually off-topic; I think it's important to have one now, but I don't want it to be a constant thing.)
Are there any obvious improvements to this scheme? Any concerns you have? Any topics that should clearly be added to the categorization above? Any features that are worth diverting development time for?
P.S. Thanks to David Zureick-Brown and Anton Geraschenko (the founders of MathOverflow) for sharing their advice, which led to this post and many of the ideas therein!