March 11, 2019
The first in a series of reports co-authored by the Network Contagion Research Institute and ADL’s Center on Extremism
The toxicity on social media creates victims online and online ecosystems that breed real-life hatred. Gab, a self-described “free speech” platform largely used by right wing extremists, has been the preferred platform for hatred and vitriol. Users include Robert Bowers, who posted on the site just before he massacred congregants at the Tree of Life Synagogue in Pittsburgh.
Social media platforms such as Twitter consider the best ways to respond to hate and extremism, including by “de-platforming” – or banning users who violate their terms of service – to remove the toxicity on their platforms. There is some debate around de-platforming and whether it solves or just suppresses hate and extremism, and also whether it reduces extremism on one platform that only resurfaces – potentially more virulently – on others.
The Network Contagion Research Institute (NCRI) is a group of researchers who have partnered with ADL to look at challenging social media analysis. NCRI director and co-founder Joel Finkelstein, who conducted the study and co-authored the report, is a COE research fellow.
This line of research includes a deep dive into how Twitter bans may serve to exacerbate Gab’s virulent ecosystem. What we found is that when Twitter de-platforms a group of users, Gab often treats those moments as recruitment opportunities (see example below).
Fringe web communities play a critical role in the dissemination of hate and extremist content -- particularly /pol (4chan’s politically incorrect message board) and Gab. At issue is whether Twitter’s solution on its platform may drive the participation and the level of animation of hatred on Gab, implying bans are a solution for one platform but could actually be a detriment to the internet as a whole – potentially reverberating into our everyday lives by exacerbating hate in our communities. While there are benefits to others on Twitter not seeing hateful content, we also should be concerned about the impact on those who migrate to another platform that is even more hateful.
For background, the below table shows the top ten months for new membership on Gab in order of magnitude and corresponding major ban events on Twitter or relevant, alt right associated political events.
NCRI compared the monthly rate of new users on Gab with the dates of notable or mass user bans on Twitter. We found a close correlation between mass bans of hateful Twitter profiles and global peaks of new memberships on Gab. While other factors, such as major political events, impact membership trends, we found that months with major Twitter bans corresponded to the highest peaks of new membership (see Figure 2, new accounts on Gab by month as a percent of total accounts).
Fig 2. A graph of monthly new accounts on Gab in terms of percentages of total accounts
We will be positioned to better review this type of data if Twitter were to share comprehensive data on these factors; for now, we have used Wikipedia’s reporting on mass bans as our source for mass bans. Within these limitations, we were able to find a direct correlation between Gab membership and mass bans. As demonstrated in Figure 3, months when a Twitter mass ban took place corresponded to more than double the percent of new members on Gab than a typical month (with our confidence in this assessment at a 95% statistical significance).
Moreover, analysis of Gab users’ use of the word “ban” amidst mass Twitter de-platforming suggests they are not only joining Gab due to the ban, but highly motivated by animosity toward Twitter’s banning. In fact, close to 3% of all comments on Gab include the word “ban” (which is in the top 0.1% of all words used on the site), suggesting a high level of interest in the concept, and notable bans on Twitter (depicted by dotted lines in Figure 4 below) predict substantial increases on the use of the word “ban” on Gab, suggesting the bans themselves motivate that animus.
Fig 4. A time series of “ban” comments on Gab. Aside from the travel ban, most spikes in ban comments occur in conjunction with notable Twitter bans.
Gab users’ anger toward Twitter is deep and widespread. To model the way bans and sentiment towards Twitter are expressed in conversation on Gab, we used word2vec, a deep learning tool that illustrates contextual associations between words over large bodies of language (e.g. a website or a book). Using these associations, we depicted the terms most closely associated on Gab with the words “ban” and “Twitter.” These graphs depict the most prominent words associated with “ban” and “Twitter.”
Figure 5. shows that among Gab users, “ban” overwhelmingly references speech, such as issues related to free speech and censorship, and “ban” often accompanies derisive comments about Facebook, Google and – most frequently – Twitter. Figure 6 below shows word association specifically around the term “Twitter” illustrates hostile attitudes towards mainstream platforms: “twatter” “fascistbook” and “goolag” appear regularly on Gab. Gab users frequently deride mainstream platforms for limiting content; that derision reflect Gab users’ deeper hostility toward mainstream social media platforms.
Fig 5. The word associations surrounding the term “ban” (blue) reference speech (red) and mainstream social media platforms (purple).
This data suggests that de-platforming on Twitter contributes directly to increased participation on Gab. Therefore, while it may solve a problem for Twitter, it shifts a version of problem to other areas of the internet. Gab is increasingly empowered by capturing disaffected former Twitter users, suggesting Twitter is a target for their anger, and that Gab’s users now occupy a more hostile environment.
There is still much research to be done to reach a firm conclusion on whether and when de-platforming is a highly valuable solution to hate on platforms, but as platform companies calculate their response to online hate and extremism, they need to consider the effects of their decisions on the broader online ecosystem.
Fig 6. The word associations surrounding the term “twitter” show that Gab users generally characterize mainstream social media platforms with hostility and associate them with bans.