(Cross-posted from my blog)
The Web was never perfect. There were always people using it to exploit others. Sometimes, it was the small-time Ebay scammer looking for a couple of dollars. Other times, it was a giant multinational trying to get you to buy things you didn't need.
At the same time, it was a wonderful place. You could meet interesting people and talk about anything that you liked. In a way, it freed us from the physical limitations of geography and paper, letting our minds wander, meet, and create things like LessWrong.
But now I believe some loosely coordinated actors threaten to change the Web for the worse. If allowed, they would turn it into a medium that restricts interaction and promotes passively consuming low quality information–like cable TV. I believe advertising and recommendation systems are driving this change.
This is Water: Online Advertising
The prevalence of ads has normalized them. Most people I've met have never asked "Where do ads come from?" If they did, they would discover it involves private data centers, a small army of software engineers, and enough hardware purchases to influence global markets. But for most, it's simply a part of reality, like electricity and supply chains.
In line with this norm, Google, along with a handful of other online advertisers, developed a new standard called Federated Learning of Cohorts (FLoC).
The way it works is this: while a user surfs the Web, their browser analyzes what content they see. Based on that, it assigns them to a cohort of users with similar preferences. Any ad system they encounter can request the cohort ID and use that to display ads targeted at a user's specific preferences.
Privacy advocates have scrutinized FLoC and found multiple problems. For example, exposing users' ethnicity, political affiliation, and other traits to anyone who asks for it, making it easier to discriminate against them. Or creating convincing scams targeted at specific groups of people. Or... I'll stop here.
However, what I haven't seen discussed is that FLoC would write advertising into a standard. That would make every browser an extension of the online advertising industry. I believe this is dangerous because it further normalizes the idea that ads are critical to the Web.
But are they?
The common argument in favor of ads says that ads are better than alternatives like paywalls and micropayments. They allow users to trade a little of their attention and bandwidth for content they would not be able to pay for if it was behind a paywall. And despite multiple attempts over the years, nobody has managed to get micropayments to work.
There are two problems with this. First, a little over 40% of users employ ad-blockers. That's a clear signal that they dislike ads. Second, ads create an environment that rewards low effort, low quality information. It's one of the reasons why web search seems to be getting worse–most results are pages generated to game ranking algorithms to earn clicks from displaying ads (or scamming people).
And yet FLoC would weave ads deeper into the Web's fabric.
Free to Choose: Recommendation Systems
Like ads, recommendation systems are everywhere. And like ads, their familiarity helps conceal their size and complexity. It also masks how much they influence us.
On a basic level, they do so by offering us a set of "good enough" choices. This way, when a user finds something they like, they can keep on getting more of the same until they exhaust the set or get bored. Because of this, they also see less content they dislike, but more importantly, they spend less time actively searching.
I believe this is making users more passive. It limits the number of decisions they make, offloading them to a program someone else wrote. Sometimes, when it acts as a tool to explore a set of media, it can be helpful. But it seems that more frequently it keeps the user engaged by pushing just the right buttons in their brain–like pages that display the best memes or most triggering news.
One could object and say that recommendation systems reveal users' true preferences. They just so happen to be: pretty or outraged faces; quick camera cuts; baby-like animals; etc.
But I feel this argument misses how much design influences our behavior. There are plenty of real world examples of this, like in urban planning.
Robert Moses shaped many parts of New York City. He created parks, highways, social housing projects, and even some bridges. His designs favored quantitative aspects over qualitative ones: throughput over well-being, capacity over community interaction, etc.
The highways he built ran through thriving neighborhoods, causing residents to move out and crime to move in. Some of the communities affected this way, like the small Swedish diaspora, never re-materialized and disappeared from the City. In a similar fashion, the social housing projects he designed made forming communities harder. That lead to a steady level of crime, trapping residents in a bad equilibrium.
By analogy, recommendation systems are architectural elements that nudge users toward passively consuming clickbait, turning the Web into a quieter, single-play-like experience.
TV-ification of the Web
Acting together, ads and recommendation systems create a feedback loop: recommendations draw people's attention, which in turn draws advertisers, whose money improves recommendations, which attract more people and so on.
This loop is beginning to change how the Web works, primarily benefiting ad-tech. In my mind, FLoC is a clear example of this because it attempts to cement ads as an integral part of the Web. Aside from the problems I already described, I think it will make it harder to experiment with alternative models of creating content, trapping us in a local maximum–a passive, ad-filled Web that's more like cable TV than anything else.
Why do I think this would be a bad future?
Neil Postman wrote a whole book about it. In "Amusing Ourselves to Death", he explained that television is a worse medium than print because it cannot sustain the same complexity of ideas as the latter can. It reduces everything to entertainment: political debates resemble game shows; commercials take the form of micro-stories; news programs–full of quick takes and music effects–are like action films.
If the Web gets simplified like that, we will lose something incredible. It will no longer be a place where minds can meet, where someone's skin color or nationality is irrelevant, a place where, despite scams and trolling, friendships and communities form and thrive.
In the language of LessWrong, a TV-like Web means lowering the global sanity waterline. That's what has me worried.