Recently, I've noticed an interesting failure mode in some communities I've observed online.

In this scenario, there would be some kind of content that appealed to certain users, but that many others wanted to avoid seeing and indeed considered undesirable (gossip/drama, angry political arguments, sexually explicit content, etc.). In order to prevent this content from "bleeding into" conversation, administrators would create new channels where this sort of thing could be quarantined away from normal discussion (or repurpose existing channels to sequester this content), thus allowing people who wanted to avoid this sort of content to avoid having to deal with it.

However, at least from my perspective this sometimes had a bit of an ironic effect. Once the "containment area" was established, there was now an obvious affordance for this type of content that did not exist previously. As a result, the quarantined content would actually become much more prevalent in the community - further, in some cases newcomers would join the community looking for one thing, see the quarantine channel, and think something along the lines of "wow, I never knew this was a community for <insert undesirable content here>!"

The net effect of this was that the quarantines often backfired; meant to isolate undesirable or controversial content, they instead made that content more prevalent in the community as a whole, even if people no longer had to encounter it in general channels.

The chief lesson I draw from this is that it can be better to simply forbid unwanted content rather than creating a quarantine for it. If there are truly people coming to your community in large part for something that many would rather not have, you may not want them there; further, there may be those who are driven away by the presence of such content, and insofar as the undesirable content presence increases the latter group may become significantly larger.


12 comments, sorted by Click to highlight new comments since: Today at 2:48 AM
New Comment

In my experience, quarantine channels are a good choice if some participants want a particular kind of content, and it's agreed to be appropriate for the community, but not everyone wants to view it. For example, a writing discord I participate in has several NSFW channels. It's agreed upon that some people might want to write and talk about writing NSFW things, and that other people don't want to view NSFW content (because of age, religion, personal preference, etc.). I think it's a bad idea to create a quarantine channel for content that is actually inappropriate for the community: for example, I banned religion as a topic in my parenting discord, instead of creating a religion channel.

For the record, I think some of the examples you have of this were instances where it was in fact actually desired that that content be welcome, and you just have values disagreements with the communities in question. but it does seem true to me that the pattern exists, which is why I'm not creating quarantine channels on a community like this that I'm running.

My friend FireBatVillain drew my attention to the following study: You Can't Stay Here, The Efficacy of Reddit's 2015 Ban Examined Through Hate Speech. He points out that this study indicates after hateful subreddits were banned from Reddit, the removal of the offending subreddits did not cause "hate speech" to increase on other parts of the site - on the contrary, even subreddits that saw an influx of users who had formerly used the banned areas did not see significant changes in "hate speech" usage.

In other words, this study shows an instance where the existence of spaces for certain types of bad content was increasing their prevalence, and removing those spaces did not cause the content to "spill back" into the rest of the site.

Now, one difference between this and my original claim is that the spaces in question were not explicitly containment areas - however, I still consider this to be relevant supporting evidence.

I've noticed the opposite effect. I'm a fairly active member on the LessWrong IRC channel (#lesswrong, on Freenode), and a year ago, in light of current events at that time, political discussions were threatening to take over the channel entirely. So, what one of the moderators did was to create a separate channel, #lw-politics, into which political discussions could be moved. The effect was immediate and dramatically positive. Political discussions are often confined to #lw-politics and when a political discussion starts up in the main channel, very often participants will notice that they're in the main channel, and voluntarily shift the discussion to #lw-politics, in order to free up bandwidth on the main channel. The end result has been a fairly clean separation of political content from more general interest rationality content.

Similarly, the SlateStarCodex subreddit has quarantined politics into its weekly "culture war" threads with similar results. In both cases separating politics from the main topic of the forum or channel led to a decline in the amount of political content and a cleaner separation between the political content and the primary topic of discussion.

I second this observation (about IRC; I do not frequent the SSC subreddit, and so cannot comment on that part).

I think the key difference here is that IRC is set up in a very different way from, say, Discord, or Slack. Consider:

If I join a Discord server or a Slack workspace, everything on that server is controlled by some particular person or people, who have chosen to set it up; everything in that place, pertains to that community. Either I am there to interact with that community, or there’s no point in being there. Likewise, there is likely great overlap between the participants of one “channel” or another, as they are drawn from the same relatively small, tightly linked (by bonds of acquaintanceship) group of people.

If, on the other hand, I connect to an IRC network, such as Freenode, I will find that there is a tremendous profusion of channels (in excess of fifteen thousand, as I recall, by a count I once took); these are set up by the greatest variety of people, from many different communities, many or most of them having nothing at all to do with one another. The median overlap between the population of regulars of any two randomly selected IRC channels on Freenode (or most other IRC networks) is zero. Consequently, there can be vast differences in local culture, norms, etc., between IRC channels on the same network.

Consider also that on Discord or Slack, each of the channels on the server/workspace have been set up, deliberately, by the leaders of the local community—and thus the existence of any channel on a Discord server or a Slack workspace implies that the community, and its leadership, endorse and approve of the existence of that channel, and of the discussions that take place therein.

On IRC, on the other hand, anyone—anyone at all, from any channel or community, or simply anyone who connects to the network, can create a new channel (simply by joining it). If, for example, you see a channel called ##BobJones, and another channel called ##BobJonesIsAWeenie, can you conclude that Bob Jones set up the second channel, or that he endorses its existence, or that he even knows about it? You cannot. (Indeed, you can’t even conclude that about the first channel—but that’s beside the point.)

Then there is the very important technical/UX difference. If I join a Discord server or a Slack workspace, I can easily and at a glance see what channels are available. I can thus immediately see a list of all the sorts of discussions that this particular community, and its leaders, endorse.

Whereas if I connect to the Freenode IRC network… well, it is possible to view a list of all channels, in principle. (Although some IRC clients will not let you easily do this.) But then what? Will you scroll through the entire five-digit-strong list of channels? How do you know which ones are related to each other? Which ones are set up by whom, endorse by whom, frequented by whom? You don’t.

Thus the structural and technical differences between different communications protocols/systems go quite a long way toward explaining these different social experiences. (And I happen to think that in this—as in everything else—IRC is by far the superior of its more modern “competitors”.)

Great points here. The UI/UX distinction is key here in my view - when the act of creating a containment space necessarily makes the existence of such a space visible, the problem I've described is much more relevant than when the existence sort of space is not immediately apparent to users - even if via simple obscurity in a huge list, as with IRC.

(One potential way to address this in Discord is to make the containment space opt-in, but in my experience this has not been particularly effective, in part because the best way to do this on Discord (roles) is itself quite easy to notice.)

My read was that the opposite was the case with r/SSC - that >50% of the activity of the whole sub-reddit is in the weekly 'culture war' thread and also that the total political content increased once it became more concentrated. I'd be interested if anyone could gather data on this.

Quickly looking it seems like there are ~3,000 comments/week on the SSC thread. I just eyeballed (rounding everything to nearest 10) the total number of comments on posts in the last week (i.e. between the last 'culture war' thread and the next) and got 1130 comments, which indicates a 3:1 ratio of political content to non-political content.

This isn't a perfect metric though, as posts are more important than comments. The total number of posts in that time period was 47, and if each is worth 10 comments, then that only brings us to a 2:1 ratio, but maybe they should be worth 100 comments, making it a 1:2 ratio (against politics).

I tried to compare the ratio to the SSC reddit era before the 'culture war' thread, but reddit wouldn't let me go back that far (after hitting a page with the 1000th article, sorted by 'new', it said 'there doesn't seem to be anything here'). So I don't know if we can draw a clear line from then to now. Also that line would be quite imperfect anyway because there are other forces that cause people to write political things, than just having a space+affordance for doing it.

Looking over exciting posts from that era being discussed on the subreddit, my weak impression is that the sub has grown quite a lot in the period from when the 'culture war' threads began weekly. Exciting posts back then seemed to have 10-50 comments on them, whereas today they can have 100s and 1000s. So I don't think the 'quarantined area' has had a marginalising effect on the content.

I take this overall as weak-to-medium evidence in favour of the OP's hypothesis. As I said, any clearer data on this forum or other forums would be appreciated.

I'm still unconvinced. Both your data and David_Kingsley's data seem to confuse correlation with causation. The rationality community has grown. It has more people interested in politics than it had in the past. This led to more politics being discussed, which, in turn led moderators to create politics-only sections on forums, discords, etc. to prevent politics talk from overwhelming the main channel and turning off those who were not interested in discussing politics. This seems to explain the increase in political posts and the quarantining of politics without requiring a causal link between the two.

And yet I now notice people saying things along the lines of "SlateStarCodex is a place to go for culture war things" and the like. If that was intended that's fine, but I think culture war stuff on SSC absolutely falls into the category where an affordance is being created.

Seconding the "SlateStar culture war seems like precisely the example of this thing happening."

But has that led to an actual increase in the prevalence of politics on /r/slatestarcodex?

So "quarantines" shouldn't be channels - they should be their own thing.