Note — Dec 20, 2020

Facebook Is a Doomsday Machine

Seen in → No.155

Source → theatlantic.com/technology/archive/2020/12/...

I'm pretty tired of Facebook (and account-less for a few months now), and thus rarely include it here, at least in featured articles. However, this one by Adrienne LaFrance is really good in the way it frames the issue and in mixing a damning case against the company, with a look back at the idea of the doomsday machine, Herman Kahn, RAND, futurism, and scenarios. LaFrance readily admits the comparison is imperfect, but the idea of not only scale but megascale and the impact of that unique size on society is a useful one.

I’d add that when people refute this type of accusation, or say the message is “alarmist,” they often roll out the research showing that Cambridge Analytica didn’t have that much impact. To my mind, those are two different kinds of influence. There is no proof that anyone can affect a large number of people towards a precise behaviour, but I think it’s pretty clear by now that FB and others do have influence in changing people’s belief at scale, it’s “simply” not precise or even purposefully directed, other than for engagement. A blunt instrument but a megascale one.

[I]t took the concept of “community” and sapped it of all moral meaning. The rise of QAnon, for example, is one of the social web’s logical conclusions. That’s because Facebook—along with Google and YouTube—is perfect for amplifying and spreading disinformation at lightning speed to global audiences. Facebook is an agent of government propaganda, targeted harassment, terrorist recruitment, emotional manipulation, and genocide—a world-historic weapon that lives not underground, but in a Disneyland-inspired campus in Menlo Park, California. […]

Megascale is nearly the existential threat that megadeath is. No single machine should be able to control the fate of the world’s population—and that’s what both the Doomsday Machine and Facebook are built to do. […]

[T]here aren’t enough moderators speaking enough languages, working enough hours, to stop the biblical flood of shit that Facebook unleashes on the world, because 10 times out of 10, the algorithm is faster and more powerful than a person. […]

In other words, if the Dunbar number for running a company or maintaining a cohesive social life is 150 people; the magic number for a functional social platform is maybe 20,000 people. Facebook now has 2.7 billion monthly users. […]

If the age of reason was, in part, a reaction to the existence of the printing press, and 1960s futurism was a reaction to the atomic bomb, we need a new philosophical and moral framework for living with the social web—a new Enlightenment for the information age, and one that will carry us back to shared reality and empiricism.