Quantifying Hate: A Year of Anti-Semitism on Twitter

It’s nearly impossible to overstate the impact social media has on our world.

That influence is often wielded in the service of good – think viral fund-raising efforts or protecting democratic movements against authoritarian threats – but it also can powerfully amplify society’s most dangerous attitudes. Racism, sexism, homophobia, religious extremism and conspiracy theories have deep roots in social media, and perpetrators have recognized and capitalized on the near-universal reach of popular platforms.

This report is an effort to gauge the prevalence of one of these destructive prejudices – anti-Semitism – on one social media platform: Twitter – by examining the one-year period from January 29, 2017 to January 28, 2018.

ADL frequently fields reports from constituents who describe an atmosphere of anti-Semitism on Twitter, including harassment from anti-Semitic trolls. We issued previous studies on this phenomenon. Media reports reflect this phenomenon as well. Experts at the ADL’s Center on Extremism (COE) wanted to go beyond those reports, to understand the problem in greater detail while examining tweets for nuance and context.  How many anti-Semitic tweets are posted on Twitter during any given week? How do we know the tweets truly are anti-Semitic?  What if some of these tweets are intended to convey an ironic or critical message?

Using a proprietary, wide-ranging query, as well as statistical methods and expert analysis, COE analysts were able to determine that roughly 4.2 million anti-Semitic tweets were posted and reposted on Twitter in the one-year period specified above. We estimate that the tweets were issued by approximately three million unique handles.

Of course, 4.2 million tweets is a very small number out of the trillions of tweets sent on the platform each year. But that does not negate the lived experience of Jews who have found Twitter to be a toxic environment.  This number is still large enough to underscore the powerful harassment that exists and the ease with which a relative handful of users can infect our shared social media environment with negative stereotypes and conspiracy theories about Jews.

This report, and the random statistical sample of over 55,000 carefully reviewed tweets which it generated, also will be useful to those designing artificial intelligence-based efforts to counter anti-Semitism and bigotry online.

ADL already works with Twitter to help them address anti-Semitism and other forms of bigotry on their platform, and we know this is a difficult challenge to solve. We are proud to be part of Twitter’s Trust and Safety Council and to partner with Twitter as a member of our Problem Solving Lab to explore engineering-based approaches to address online hate and harassment. We already have seen the company make real progress on these issues and demonstrate leadership among social media platforms.

And yet, despite recent advancements, intolerance on public platforms like Twitter still endures. It degrades the conversation and poisons the public discourse for all. Even a proportionally small number of Tweets that express anti-Semitism is too many, and any platform that hosts such hate must ensure that it is using all means necessary to tackle the situation even as it seeks to balance legitimate concerns about not inhibiting freedom of expression. For this reason, this report includes not only a detailed review of our findings but a set of recommendations for Twitter to take into consideration as it attempts to address the issue and improve its performance.

We look forward to continuing to work with Twitter – and other platforms that seek our help – to ensure these environments are a safe space for all users, regardless of their faith, race or other immutable attributes.

As a result of a pilot research project, the Anti-Defamation League can estimate that a minimum of approximately 4.2 million English language anti-Semitic tweets were disseminated between January 29, 2017 and January 28, 2018.The analysis showed that within predefined seven-day periods, the number of tweets expressing anti-Semitic sentiment ranged from a low of 36,800 in week 26 (July 23-29) to a high of 181,700 in week 45 (December 03-09). The average number of tweets expressing anti-Semitic sentiment throughout all 52 weeks of the analysis was 81,400.

1. Process

The current findings are based on a complex Boolean query designed to identify language frequently used by anti-Semites. The query was broadly written to encompass obvious expressions of anti-Semitism, including classic anti-Semitic stereotypes; code words and symbols sometimes used in an anti-Semitic fashion; and also subtle references to anti-Semitic conspiracy theories.

Experts at ADL’s Center on Extremism assessed statistically significant samples of the tweets captured by this query for each week. The assessments were designed to identify the percentage of tweets not intended to express anti-Semitic sentiment or disseminate anti-Semitic conspiracy theories. Of particular interest were tweets that used or cited anti-Semitic language in order to condemn it, or that used anti-Semitic language ironically. In these cases, analysis of the thread context (as well as accompanying videos and images) was critical. Occasionally, the self-reported info in the tweeter’s “bio” field was also consulted to help determine intent. When ADL experts were not reasonably confident that a given tweet was anti-Semitic, it was marked as not containing anti-Semitic sentiment. A sufficient number of tweets were assessed for each time period to result in a 3% margin of error.

The resulting percentage of anti-Semitic tweets in the representative sample was then extrapolated back to the entire population of tweets pulled in by the query for its time period. This yielded the number of anti-Semitic tweets reported in the findings.

This project is ADL’s second attempt to assess the amount of anti-Semitism on Twitter. In an October 2016 report, ADL used keywords correlating with anti-Semitism to assess the total number of tweets which could potentially contain anti-Semitic content that were directed at journalists. The current report refines the methodology by moving beyond simple keyword detection and estimating the number of instances of actual anti-Semitic sentiment or content. The current report also expands the scope of the first study to address anti-Semitism in all English language tweets.

A third project, the Online Hate Index being developed by ADL’s Center on Technology and Society, has experimented with artificial intelligence and machine learning to detect the presence of hate speech on selected Reddit forums. The current project focuses solely on anti-Semitism, including difficult-to-detect anti-Semitic conspiracy theories, and was designed and implemented by human experts without any automation or machine learning. ADL hopes that the project’s resulting corpus of tweets — which have been evaluated for anti-Semitism — will be useful in training machine learning engines to detect anti-Semitism.

2. Defining Anti-Semitism

The current report is based on a proprietary Boolean query designed by experts at ADL’s Center on Extremism. The query is designed to detect anti-Semitic content in the following categories:

1. Classic anti-Semitic stereotypes (e.g. references to Jews as greedy; controllers of banks, media, governments and academia; under-miners of culture and racial purity; cursed for killing Jesus; etc.)
2. Positive references to or promotion of known anti-Semitic personalities, authors, books, articles, videos and podcasts
3. References to anti-Semitic conspiracy theories (e.g. Jewish control of the Federal Reserve; the existence of a “Zionist Occupation Government,” etc.)
4. Holocaust denial
5. Epithets used for Jews (e.g. “kike”) and against Jews (e.g. “goddamn Jews”)
6. Code words and anti-Semitic symbols such as the “echo symbol” (“((( )))”)

The current report includes criticism of Israel or Zionism when such criticism makes use of classic anti-Semitic language or conspiracy theories, or when it ascribes evil motivations to significant numbers of Jews. General criticism of Israel or its policies is not counted as anti-Semitism. [See below — When Anti-Zionism is Anti-Semitic.]

3. Caveats

The current study is concerned with all instances of tweets intended to convey anti-Semitic content, including retweets. However, cases where anti-Semitic tweets were retweeted for the purpose of condemning them were excluded by the study, as described above.

Because the query was run on Twitter data captured in early January 2018, it does not include a large number anti-Semitic tweets that were deleted by their owners or from accounts previously shut down by Twitter for violating its terms of service. As a result, the actual number of anti-Semitic tweets is undoubtedly larger than the number estimated in this study.

By the same token, the data for this study was supplied before the so-called “Twitter Purge” of late February 2018, when Twitter deactivated thousands of accounts associated with automated bots. It is possible that some anti-Semitic tweets were promulgated by such bots, and so a new search of Twitter might identify fewer anti-Semitic tweets than reported in this report.

The Boolean query was text-based, so tweets containing anti-Semitic imagery without anti-Semitic text would not have been pulled in. This is another reason we believe the actual number of anti-Semitic tweets is larger than what is estimated in this study.

1. Results of the Analysis

The query for this study was designed to be broad enough to return all results that could conceivably be anti-Semitic. This is why subsequent evaluation by human experts was so important (i.e. to weed out all of the non-anti-Semitic tweets that we knew would be pulled in). From January 29, 2017 to Jan 31, 2018, our query pulled in almost nineteen million tweets, with weekly figures ranging from 183,000 (week 30: July 23-29) to more than one million tweets (week 33: Aug 13-19).

We then subjected a random, statistical sampling of more than 1,000 tweets per week (more than 55,000 over the course of the year) to manual analysis for the presence of anti-Semitic sentiment or the dissemination of anti-Semitic conspiracy theories. Our sample sizes provided a margin of error of three percent. A total of 4.2 million anti-Semitic tweets was detected, with weekly totals ranging from a low of 36,800 in week 26 (July 23-29) to a high of 181,700 in week 45 (December 03-09). The average number of tweets expressing anti-Semitic sentiment throughout all 52 weeks of the analysis was 81,400.

The percentage of tweets pulled in by a query which tested positive for anti-Semitism ranged from a low 8.9% in week 33 (August 13-19) to a high of 34.2% in week 18 (April 30-May 06). The average percentage of anti-Semitic tweets in the context of the broad query was 23.5%.We estimate that approximately three million unique Twitter handles were responsible for the 4.2 million tweets.

2. Trend line

Although the trend line of anti-Semitic tweets rises over the course of the 52 weeks surveyed in this study, there is no obvious inflection point or detectable reason for the increase.

3. Drivers of anti-Semitic Content

Current events were reflected in the anti-Semitic tweets almost as soon as they were reported. However, because our methodology does not allow us to reliably track tweets’ thematic content over a particular time period, we can’t know the extent to which the total volume of anti-Semitic tweets is affected by current events.

Anti-Semitic tweets also reflect the ebb and flow of anti-Semitic content on other social media platforms. A significant number of anti-Semitic tweets were shares or discussions of anti-Semitic content from YouTube, Reddit, 4Chan, or conspiracy-oriented websites and blogs. The current study does not allow us to assert with statistical certainty how much the total volume of anti-Semitism on Twitter is related to anti-Semitic or conspiracy-oriented materials on other online platforms. Nonetheless, our findings underscore the fact that social media platforms are not closed systems. Further research may be able to more precisely map out the channels of anti-Semitic influence between and among social media platforms and websites.

4. Deletion of Posts

In the course of evaluating the Twitter data for anti-Semitic sentiment, it became clear that a non-negligible percentage of anti-Semitic tweets had been removed from Twitter after they were downloaded for our study. In some cases, the tweets were deleted by their users, but often they were deleted by Twitter as part of its enforcement of terms of service violations. Future studies may attempt to determine the number of anti-Semitic tweets that had been subsequently removed from the platform.

5. Noteworthy Weeks

• Week 29 (August 13-19) — Charlottesville:Interestingly, week 29 of the study — immediately following the “Unite the Right” rally in Charlottesville, Virginia — had the highest number of possible anti-Semitic tweets but the smallest percentage of tweets that were actually anti-Semitic in nature.  In the days after the rally, Twitter exploded with posts about the anti-Semitism and racism on display at the rally — which is why such a large number of tweets that were potentially anti-Semitic were pulled in by our query. Human analysis showed that 8.9% — or 92,400 — of those tweets actually promoted anti-Semitism. Despite this ratio, the sheer volume of tweets that week resulted in the 13th-highest number of actual anti-Semitic tweets in all the weeks covered by our study.
• Weeks 45 (December 03-09), 42 (November 12-18), and 52 (Jan 22-28): The highest numbers of anti-Semitic tweets were recorded in weeks 45 (181,700 anti-Semitic tweets), week 42 (146,500 anti-Semitic tweets), and week 52 (145,500 anti-Semitic tweets). Of the three, the only week in which we could discern a pattern was week 45, which was the week that President Trump announced that he was recognizing Jerusalem as the capital of Israel and that he intended to move the United States embassy there. “Trump” and “Jerusalem” appeared in our results a significant number of times that week. No discernible pattern emerged from a closer analysis of the anti-Semitic tweets in weeks 42 and 52.

As stated above, our current methodology does not allow us to determine with statistical certainty the particular themes that surfaced in each week. What follows are impressionistic observations based on the tweets that surfaced consistently in our samples over the duration of the study.

Anti-Semitic commentary erupted immediately after the New York Times’ October 5, 2017 report about Harvey Weinstein’s decades-long history sexually abusing and harassing women. The story touches on multiple long-standing anti-Semitic tropes, including Jewish control of Hollywood and the media, and Jewish sexual degeneracy and perversion.

Other tweets used the Weinstein scandal to share their views on alleged Jewish dominance of Hollywood.

As the #MeToo movement gathered steam and other men were implicated, threads were quick to note which men were Jewish.

Explanatory Note: “KhazarMafia” is a reference to the theory that many modern Jews are descendants of the Khazars, an Eastern European nation which allegedly converted to Judaism in the eighth century. Although this theory is not inherently anti-Semitic, anti-Semites have repackaged it as the “Khazar hypothesis,” which claims that these Khazar converts and their descendants have a secret agenda to undermine other societies and enrich themselves at the expense of their non-Jewish neighbors. This anti-Semitic conspiracy theory is also attractive to some anti-Zionists who believe that it undermines the historical claim of modern Jews to the land of Israel. In this instance, the Twitter user is using “KhazarMafia” as a code word for “Jews.”

A longstanding anti-Semitic conspiracy theory posits that a cabal of Jews led by members of the Rothschild banking family have been manipulating currency and exerting influence on regional and national events since the early 19th century for the purposes of personal enrichment and/or world domination. Anti-Semitic conspiracy theorists claim that the Rothschilds interfered with the American Revolution, financed Nazism and/or the Bolshevik Revolution, perpetrated the 9/11 attacks, and created the state of Israel. They also allege that the Rothschilds are some of the chief architects of an impending “New World Order” which will abolish nationalities and enslave humanity.

Over the course of this study, a significant number of tweets incorporated current events into Rothschild conspiracies: The Rothschild clan allegedly caused the disappearance of Malaysia flight 370 in order to take control of a semiconductor business.

The Rothschilds, who allegedly view Vladimir Putin and/or Iran as some of the foremost obstacles in their effort to establish a New World Order, gave a grant to the McCain Institute for International Leadership at Arizona State University in order to buy John McCain’s support for the “Russiagate” investigation and/or to support Saudi Arabia against Iran.

The arrest of numerous figures in Saudi Arabia is viewed as an attack on the Rothschilds and their goal of world domination. According to the cited article below, the “House of Saud are all descendants of Arabian Jews,” who are aligned with the Rothschilds.

As noted above, this report distinguishes between anti-Semitism and criticism of Israel, its actions and policies. Sharp criticism of Israel, even criticism that pushes the limits of civility, is not necessarily anti-Semitic. However, anti-Semitic tweets will often employ the term “Zionist” as a stand-in for “Jew,” in a context that clearly indicates the tweet was either motivated by anti-Semitism or perpetuated anti-Semitic themes.

In some following cases, the link is explicit:

In other cases, the term “Zionist” was used in a manner that incorporated classic anti-Semitic language or themes, including allegations that Jews hate non-Jews and control the media and financial and cultural institutions.

Also apparent in the tweets: the anti-Semitic notion that Zionism is either the cause of Nazism or an expression of Nazism. Although our study excludes cases where specific Israeli policies or practices are described as Nazi-like, we included cases where Twitter users described Zionism as a movement as racist or having “Nazi” characteristics.

One of the most interesting social media phenomena in 2017 was the rise of “QAnon” conspiracy theories. In late October 2017, an anonymous user known as “Q” began posting batches of cryptic comments which were interpreted by readers as secret intelligence from the Trump administration about actions it was taking to oppose an anti-American conspiracy. As followers attempted to deduce the events and actors referenced in the QAnon comments, social media erupted with a multitude of conspiracy theories. On Twitter, more than 3.3 million posts referencing QAnon appeared between October 2017 and January 2018.

The vast majority of QAnon-inspired conspiracy theories have nothing to do with anti-Semitism. However, a small percentage of tweets referencing QAnon also referred to Israel, Jews, Zionists, Rothschilds (a Jewish banking family at the center of numerous anti-Semitic conspiracy theories), or George Soros (a progressive Jewish philanthropist who is often demonized by the political right).  This study’s methodology does not allow us to determine how many of the QAnon tweets containing those terms actually expressed anti-Semitic sentiment, but an impressionistic review revealed some troubling examples.

A purported scheme pertaining to the Jewish Rothschild family centers on the collision of a small airplane and a helicopter on November 17 near the town of Upper Winchendon in England, killing four people. Also nearby was Waddesdon Manor, an estate owned by the Rothschild family. Conspiracy theories pertaining to the Rothschilds immediately erupted. many of them drew on recent QAnon material, which stated in part, "we know whe[R]e you/the family are at all times."

Conspiracy theorists concluded that the capital R was a veiled reference to the Rothschild family and that Q had "predicted" the plane crash.

Although the amount of anti-Semitism in QAnon-related content is currently very low, it has the potential to proliferate especially quickly given the viral nature of the subculture.

A noteworthy anti-Semitic theme in 2017 was the use of the term “globalist” as an anti-Semitic slur. Although the term is not inherently anti-Semitic, “globalist” is often used as a pejorative term for people whose interests in international commerce or finance ostensibly make them disloyal to the country in which they live, or who are willing to undermine the financial security of their neighbors in order to benefit transnational interests. Because of the long history of anti-Semitic associations of Jews with money and commerce, and allegations that Jews place their transnational ethnic affiliations ahead of the interests of their non-Jewish neighbors, these pejorative subtexts quickly take on anti-Semitic connotations when the term is applied to individual Jews, groups of Jews, or places where Jews are known to live (i.e. “New York globalists”). Anti-Semites frequently use the term “Globalist” as a code word for Jews.

Holocaust denial is an anti-Semitic conspiracy theory alleging that Jews fabricated the facts of the Holocaust, including that six million Jewish people were murdered by Nazis, many in gas chambers.  Because the truth of the Holocaust is so well-known, Holocaust deniers either implicitly or explicitly promote the anti-Semitic claim that Jews have managed to influence governments, the media, and academia to parrot and embed this fake story in the collective consciousness. Anti-Semites claim that Jews created the Holocaust in order to garner sympathy, to establish the state of Israel, to acquire reparations money, and/or to demoralize Germany and white Europeans in general.

As is the case with many other types of anti-Semitism, some Holocaust deniers make a point of attacking “Zionists” instead of Jews, but the anti-Semitic implications remain the same. See: When Anti-Zionism is Anti-Semitic

In April 2017, some Holocaust deniers latched on to (then-White House press secretary) Sean Spicer’s incorrect claim that Hitler never used poison gas against his enemies. Although Spicer quickly clarified that he was not referring to the death camps in which Jews were killed, Holocaust deniers stuck to their own script.

Anti-Semitic conspiracy theorists have long blamed the hidden hand of the Jews for many of the world’s worst tragedies and disasters. But while traditional conspiracy theories usually allege that Jews obtain some direct material benefit from the plots they purportedly engage in, “false flag” conspiracy theories accuse Jews of concocting events for no other reason than to gain sympathy or to provoke a favorable response.

Simple false flag theories are sometimes raised when synagogues or other Jewish institutions are vandalized. False flag conspiracy theorists often claim that the vandalism was committed by Jews themselves in order to stoke fears of anti-Semitism, thereby making Jews more sympathetic, allowing the synagogue to secure enhanced police protection, and increase Jewish clout in the realm of public opinion.

In false flag conspiracy theories, as in many other expressions of anti-Semitism, it is also common for blame to be placed on the state of Israel or “Zionists,” instead of Jews. Elaborate fantasies are constructed to show that particular events, which may not be directly related to Israel, were actually the result of Israeli manipulation (or the manipulation of Israel’s supporters and/or Zionists) to obtain some kind of less-than-obvious benefit, such as to undermine Israel’s geopolitical enemies or to promote war among non-Jews.

Among the more bizarre false flag conspiracy theories that persisted in 2017 is the claim that Israel, Zionists, or Jews are actually responsible for creating ISIS:

Jews, Zionists, or Israel are held responsible for other terrorist attacks around the world:
 

False flag conspiracy theorists also claim that Jews are responsible for many of the recent mass shootings in the United States, “explaining” that Jews orchestrate mass shootings in order to gin up political support for gun control.

In mainstream conservative discourse, Hungarian Jewish billionaire, Holocaust survivor and philanthropist George Soros is often portrayed as a potent force behind liberal and progressive causes. In far right circles, this belief often veers into outsized conspiracy theories, including claims that he directly funds specific protests or in other ways manipulates events to further his goals. These conspiracy theories sometimes take on an anti-Semitic cast, especially when they associate Soros’ actions with his Jewish identity, incorporate elements of classic anti-Semitic tropes like control of governments, banks and media, or accuse him of orchestrating false flags events (as discussed above).

A noteworthy false flag allegation against Soros claims that he was responsible for the deadly Unite the Right rally in Charlottesville, VA.

Other tweets refer directly to Soros’ Jewish heritage in pejorative terms, claim that he is trying to undermine western civilization, or following a long-standing pattern of Jewish behavior.

We are proud to be part of Twitter’s Trust and Safety Council and to work with Twitter as a member of our Problem Solving Lab. We hope that the company will continue to refine its approach. We appreciate that Twitter is making efforts to improve conversational “public health,” and to better understand how their platform impacts society. The steps that the company already has taken to date are encouraging. And yet more can and should be done to ensure Twitter, like other social media platforms, is a safe space for all its users regardless of their faith or other attributes.

These recommendations flow from this report and are specific to Twitter. However, these suggestions are consistent with steps that we would suggest for other social media platforms. They reflect a general approach that we believe is necessary across the tech industry and our broader society.

• Explore Avenues for External Review and Input: Twitter has already taken steps to improve transparency and accountability beyond the platform, by engaging with academia and civil society. This is an encouraging start, but more is required. Giving more independent researchers and members of civil society more access to the platform’s data and collaborating to build forms of formal external accountability will help Twitter to build public trust in their products and services, and the same is true for other social media platforms as well.
• Ensure a comprehensive Terms of Service that is adequately enforced: Twitter should continue to improve its terms of service to screen out hateful content. Importantly, these terms must be enforced effectively. Based on the findings outlined in this report, it is clear that there is a considerable amount of additional work to be done in this area.
• Do not depend only on account removal: The removal of hateful content is an important part of reducing the spread of anti-Semitism and other forms of hate. In particular, bots that promote hate should be removed from the platform. However, the problem of anti-Semitism is not limited to one platform, nor is it limited to online behavior. This means any approach to counter hateful messages online must involve whole-of-society engagement, including civil society organizations.
• Use artificial intelligence to enhance efforts to identify hateful content: This report demonstrates that human involvement — to detect irony or sarcasm, for example — is critical to any comprehensive review and understanding of anti-Semitic content. However, artificial intelligence (AI) remains a useful tool in helping content reviewers to identify hateful content that may violate platforms’ terms of service. Machine learning can allow AI to integrate insights and develop more capable understanding of how to sort out truly hateful content.
• Ensure users have an effective option that decreases the chances they encounter hate speech: Google’s “safe search” function filters out offensive content for children and other audiences. While Twitter’s offensive content filter is a start, it typically applies only to violent imagery. Twitter should expand its functionality to allow users to screen out hateful and extremist propaganda.

Response from Twitter

ADL shared an advance copy of this report with Twitter; the company submitted the following statement:

In the last 16 months, Twitter has made more than 30 individual changes to its product, policies, and operations, all with the goal of making Twitter safer, and improving information quality. These include:

• Making affiliation with Violent Extremist Groups against its Terms of Service
• Making hateful imagery harder to find by labeling it as sensitive content and banning it from profile and header pictures.
• Communicating its rules more clearly to violators
• Improving how Twitter reviews abuse reports filed by third-party witnesses.

Additional information about Twitter's policies, enforcement and partnership with civil society groups like ADL can be found at safety.twitter.com.

ADL notes that Twitter is unique among large social media platforms in that it allows its public data to be searched by researchers through a free Application Programming Interface (API). In addition, many steps the platform takes to reduce the visibility of abusive content through safe search and a curated news feed are not visible via the API.