Campus Antisemitism Report Card™: Methodological Appendix
Based on ADL’s campus climate research and on consultations with a panel of experts, ADL researchers designed a short survey to better understand what Jewish college students consider to be most important when it comes to making campuses safe and welcoming for Jewish students. This survey, fielded by College Pulse in February 2024, asked 160 current Jewish college students at 97 schools across the US a series of questions broadly measuring the following three things: 1) the level of importance students ascribe to various policies and procedures related to Jewish students; 2) the level of importance of various types of Jewish life on campus; and 3) how concerning different types of antisemitic and problematic anti-Israel incidents on campus would be to them as Jewish students. All respondents were asked to rank how important or concerning each item would be to them as a Jewish college student on a five-point scale, with lower ratings indicating less concern/importance and higher ratings indicating that the student viewed that item as more concerning or important. In this way, gathering perspectives from Jewish students at schools across the country informed and contextualized the weightings ADL assigned.
Though the students themselves formed a non-representative sample, they represented a wide variety of universities—large and small, public and private—across the country, increasing researcher confidence that a fuller breadth of Jewish student experience was captured as a part of this survey effort. Representation of public and private universities was roughly equal across the sample, with students from public four-year universities making up 53% and students from private four-year universities at about 46%.
Informed in part by student responses to the survey, ADL researchers then devised a weighting scheme to be included in the tool to determine each university’s overall “grade” in addressing campus antisemitism and improving campus climate. First, researchers conducted an exploratory factor analysis of all responses to understand the nature of the data and how each question was correlated with underlying constructs related to campus climate for Jewish students. As expected, this analysis showed that questions broadly clustered along categories related to policy/administrative responses, on-campus incidents, and Jewish life on campus, which informed the three-category subscore approach. It is worth noting that more targeted incidents of antisemitism, including physical assault/threats and interpersonal instances of anti-Jewish prejudice, were closely related to each other, while events featuring problematic anti-Israel sentiment or content tended to cluster together. In recognition of both the qualitative and quantitative differences between these two types of on-campus incidents, each was weighted slightly differently in accordance with students’ stated concern about these incidents.
To further inform the weighting, average responses to each question were taken across the sample, with those questions having higher averages being weighted more heavily in the overall grade than those questions with lower averages. For example, because students on average ranked their school’s condemnation of antisemitic incidents as being more important (4.08 on a five-point scale) than their college taking a stance on the Boycott, Divest, Sanction (BDS) movement (3.14 on a five-point scale), the former was assigned a higher weight in the overall grade than the latter.
Generally speaking, questions with average responses indicating that the policy or incident was at least “very” important or concerning (4-5 on the five-point scale) were assigned individual weights of 15-20% each, while those averaging between “somewhat important” and “neither” (2-3 on the five-point scale) received lower weights of 5% or less. ADL’s panel of internal and external experts reviewed these ratings to ensure that weights comported with qualitative expectations of which categories are more important or egregious relative to others.
Because the three broader measures of policies/procedures, antisemitic incidents, and Jewish campus life are distinct categories each contributing varying levels of importance to a student’s overall perception of campus climate, each category was assigned a rating using the same approach as outlined above. Each individual question within that category was first weighted relative to the other questions to create a subcategory score. Based on student responses to the level of importance of each of these three subcategories, both policies and antisemitic incidents were assigned a weight of 31.5% of the total score, with Jewish campus life assigned a weight of 27%. The remaining 10% came from a qualitative assessment to guarantee that factors not accounted for within our three categories were also taken into consideration. This assessment delved specifically into the enforcement of college and university non-discrimination policies to certify that colleges that had gone above and beyond in protecting and supporting their Jewish students following incidents were rightfully rewarded and that colleges that had entirely failed to uphold their policies were graded accordingly. To ensure a rigorous qualitative assessment, the Report Card team re-reviewed primary and secondary research materials and consulted with internal Regional Directors to acquire perspectives from those on the ground. For the qualitative assessment, colleges that had exceeded expectations were rewarded, colleges that were performing at an average level did not experience any changes to their grades, and colleges that had failed expectations received downward adjustments to their overall grades. Each university’s qualitative score was subject to an intercoder reliability check, with agreement by at least three reviewers required in order to assign a score.
Based on scores in the three substantive subcategories (which contained 21 sub-criteria) and the qualitative category, each school was assigned a final percentage grade ranging from 0-100%.
In addition to being informed by the data and easily replicable, this approach to grading has a variety of advantages. First, it allows for the consideration of meaningful administrative policies and procedures that have been fully or partially enacted, while also accounting for antisemitic incidents on campus and providing due credit to universities that have fostered robust Jewish communal life and culture on campus. At the same time as it provides a useful benchmark for comparison to other schools in the current sample, this grading strategy sets a baseline that universities can refer to year-over-year as they seek to improve their commitment to fighting campus antisemitism and creating a safe, welcoming, and equitable learning community for Jewish students.
Incident Scoring
In accordance with both the importance that surveyed Jewish students ascribed to on-campus antisemitic and hostile anti-Israel incidents and the seriousness of such events, the grading criteria was arranged such that a substantial proportion of each institution’s overall grade was determined by the rate and severity of antisemitic incidents reported on each campus. To allow for differentiation based on relative severity, incidents were grouped into the following two categories, with separate grades assigned for each category:
- Severe incidents: Physical assaults, threats of physical harm, incidents perpetrated by faculty or staff, serious property damage of Jewish spaces on campus, and repeat targeted harassment;
- Other incidents: All other incidents that do not fall into the first category, such as verbal slurs against Jewish students, the use of problematic anti-Zionist rhetoric during on-campus protests, incidents of vandalism (e.g., a swastika drawn in a bathroom), etc.
Researchers drew upon two sources of event count data to determine the number of each type of incident that occurred on each campus in the sample throughout 2023 and up to late March 2024. The ADL Center on Extremism (COE) regularly tracks antisemitic incidents on college campuses throughout the US, including those incidents reported to ADL. In addition to drawing on this data, researchers reviewed the AMCHA Initiative’s antisemitic and anti-Zionist incident reporting data and news media. To avoid event double-counting, researchers reviewed event details from both sources and excluded any duplicated events from the overall count.
Per capita incident rates—both severe and other—were then calculated based on Jewish student population for each school. Resulting incident rates reflected the numbers of serious and other antisemitic incidents per 100 Jewish students during the study period. Schools with no incidents received scores of 100% in the incident subcategories, while those with at least one incident were assigned grades on a scale depending on their place in the overall distribution of scores.
Researchers divided incident rates into tertiles, with those in the lowest third of the distribution assigned a low incident rate, those in the second third assigned a medium, and those in the highest third assigned a high. This approach allowed researchers to assign incident scores based on number of incidents and Jewish student population data for the schools examined, rather than assigning arbitrary thresholds for what constitutes a low, medium, or high number of incidents.
PLEASE NOTE: This analysis combines objective data with certain subjective impressions and analysis of that data as well as our beliefs about how to weight different factors. Reasonable people may disagree with these decisions. We invite you to offer your perspective and points of view. While we strive to be accurate with our data points, please feel free to offer corrections or nuance at campus@adl.org.
1 Schools with zero incidents were excluded from the sample prior to calculating tertiles.