Related Content
When he testified before Congress last month, FacebookCEO Mark Zuckerberg discussed the problem of using artificial intelligence to identify online hate speech. He said he was optimistic that in five to 10 years, “We will have AI tools that can get into some of the linguistic nuances of different types of content to be more accurate in flagging content for our systems, but today we’re not just there on that.”
As an expert on hate speech who recently developed an AI-based system to study online hate, I can confidently say that Zuckerberg is both right and wrong. He is right that AI is not a panacea, since hate speech relies on nuances that algorithms cannot fully detect. At the same time, just because AI does not solve the problem entirely doesn’t mean it's useless.
In fact, it’s just the opposite. Instead of relying on AI to eliminate the need for human review of hate speech, Facebook and other social media platforms should invest in intelligent systems that assist human discretion. The technology is already here. It can not only help tech companies deal with the scale of this challenge; it can also make platforms more transparent and accountable.
At its core, AI identifies patterns in data sets. In his testimony, Zuckerberg may have been trying to say that AI is not a good mechanism by itself to remove hate speech. That’s true. Even the best filters will not replace human reviewers.
This is because hate speech evolves. For example, Shrinky Dinks are plastic toys from the 1980s that are designed to get smaller when baked in an oven. Toys by themselves certainly aren’t hate speech. But when those same words are used to describe Jews, as they are today by some white supremacists, the name of a child’s plaything can be transformed into an offensive Holocaust metaphor. Another example came in 2016 when white supremacists started putting triple parentheses around Jewish people’s names on Twitter in an effort to harass and intimidate them.
Imagine trying to build an artificial intelligence that could capture this subtlety. The technology simply doesn’t exist yet. Because hate speech is nuanced, even the best AI can't replace human beings. Computation will not solve the hate speech dilemma.
The clearest proof that AI alone can’t solve hate speech is the false-positive problem. As Zuckerberg explained in his testimony, “Until we get it more automated, there’s a higher error rate than I’m happy with.” However, even if AI was 99 percent effective at removing controversial content like hate speech, there would still be real consequences, made worse by the immense scale and reach of online platforms.
Take the example of terrorist propaganda: Facebook already relies on AI to tackle that. In February, the British government announced it would use AI to filter out extremist content on social media and claims its automated tool can detect 94 percent of Islamic State propaganda with 99.995 percent accuracy. With a false positive rate of 0.005 percent, if the tool analyzed 1 million randomly selected videos on YouTube, only 50 of them would require additional human review. That means 50 harmless videos that, without additional human review, would be falsely flagged as potentially criminal.
But Facebook produces much more than 1 million pieces of content every day. Facebook users produce more than 350 million photos per day. At a false positive rate of 0.005 percent, that’s 15,000 falsely flagged pieces of content every day. This equation only looks at photos on one social media platform—only showing us part of the massive problem.
These misidentifications could result in users being wrongfully removed from platforms without transparency or a right of appeal before the platform takes action. Left to run on its own, AI in this context does indeed seem like it creates more problems than it solves. But what if AI wasn't tasked with solving this problem alone? Platforms cannot tackle the hate speech challenge until they understand what hate speech means to their users and how it functions in online environments. AI is an excellent way to study how hate speech functions and evolves in online spaces.
The Anti-Defamation League’s Center for Technology and Society, in partnership with UC Berkeley’s D-Lab, created the Online Hate Index. This system uses artificial intelligence to identify hate speech based on whether a target of that speech would consider it hate speech. This is not designed to filter out content, but rather to help us understand how hate speech operates on social media. The Online Hate Index relies on hand-coding of hateful speech from humans, laying the foundation to help solve a problem that has been inadequately addressed through reliance on platform users to report instances of abuse and violations of terms of service agreements.
As a society, we don’t want social media platforms unilaterally deciding what is, or is not hate speech. But user voices matter, and consumers want tech companies to be responsive to their concerns. By using AI to identify patterns in what users consider to be hateful speech, outside of the content of the speech, mechanisms like the OHI can help platforms enhance their ability to find hate speech the way their users see it. This won’t be a silver bullet—taking action may still require nuanced analysis by human reviewers—but that combination of human and machine can both help social media platforms operate at scale, and help it get it right.
In other words, even if AI isn’t the solution on its own, executives like Zuckerberg shouldn’t be so quick to discount it at present. The best technology to combat hate speech should help human reviewers, not displace them.
Finally, if Facebook is serious about its commitment to being more transparent, it needs to share more data with outside researchers who study hate speech, as Twitter has recently done.
Hate speech is one of the most complicated problems faced by internet platforms today. At the same time, it is a phenomenon that must be addressed with a comprehensive approach if we want to truly make online communities safer and more inclusive for all people. By pairing artificial intelligence with human understanding, social media companies can foster an environment that is less prone to host hate speech and more welcoming to all people.