Does your last name commence with a letter positioned towards the latter part of the English alphabet? Beyond the potential disadvantage of prolonged roll calls, it appears you might be more prone to receiving diminished academic scores and unfavorable commentary on your submitted work.
These are the conclusions drawn by a research group affiliated with the University of Michigan, following an extensive analysis encompassing over 30 million student assessment records. The data was compiled from submissions and evaluations processed through a learning management system, specifically the platform known as Canvas.
The investigation revealed that this sequential grading bias is prevalent across the entire university, irrespective of the academic discipline. However, its impact is most pronounced within the fields of social sciences and humanities, affecting subjects such as engineering, science, and medicine to a lesser extent. A potential explanation for this divergence lies in the inherent nature of assignments in social sciences, which often allow for greater interpretive latitude, consequently presenting more grading complexities. In contrast, engineering, science, and medicine typically involve subjects with more definitive and precise answers, simplifying the evaluation process.
While this observable trend is associated with alphabetization primarily due to the default configuration of Canvas, the study underscores a distinct underlying pattern: the caliber of grading deteriorates as evaluators progress through their assigned tasks.
Students whose surnames begin with the letters A through E, and thus were assessed earlier in the sequence, attained scores that were, on average, 0.3 points higher (out of a possible 100) compared to instances where the grading order was randomized.
Conversely, students with surname initials falling between U and Z experienced a reduction in their grades by approximately 0.3 points when assessments were processed alphabetically, in contrast to a randomized evaluation sequence.
However, the influence of order extended beyond mere alphabetical arrangement. The initial ten assignments evaluated generally garnered scores around 3.5 points per 100 higher than those graded from the 50th to the 60th position.
Furthermore, among the limited cohort of graders who adopted a reverse alphabetical order (Z-A), the surname initial bias was nearly inverted, resulting in students with A-E surnames being disadvantaged compared to their U-Z counterparts.
Before attributing these outcomes to instructor bias, the researchers emphasize that a critical contributing factor is grader fatigue. It is not unexpected for an evaluator to experience exhaustion and a potential decrease in leniency as the marking period draws to a close. In numerous locations globally, academic professionals face challenges related to excessive workloads, inadequate remuneration, and insufficient job security.
“Our hypothesis leans towards fatigue being a primary driver of this phenomenon, as prolonged engagement with a task can lead to diminished attention and a decline in cognitive faculties,” states computational social scientist Jiaxin Pei, as reported in a related context.
The research team, led by Wang, proposes that educational institutions might consider augmenting their grading staff to alleviate the burden on existing personnel and implement cross-validation procedures. Such measures could effectively mitigate the potential repercussions of evaluator fatigue.
To ascertain whether the observed grade disparities corresponded to a reduction in grading accuracy, the researchers meticulously examined the qualitative feedback provided by graders, alongside student inquiries and requests for re-evaluation. This analysis aimed to eliminate the possibility that graders were simply refining their error detection skills as they proceeded through the grading process.
Their findings indicated that the comments accompanying assignments graded later tended to be more critical and less courteous. Furthermore, outcomes for these later-assessed assignments were more frequently subject to contention.
This observation “provides direct evidence that the quality of assessment is indeed lower for assignments graded later in the sequence and for students whose surnames appear later in the alphabet,” the study’s authors assert.
Given that Canvas holds the position of the most extensively utilized learning management system worldwide, and its default setting for organizing student submissions for grading is alphabetical order, the researchers suggest that the most straightforward remedy for this sequential bias would be to establish randomized ordering as the default configuration.
“The inherent design of the learning management system’s architecture inadvertently transforms individual-level grading biases into a widespread disadvantage for students bearing surnames that fall later in the alphabet,” the researchers conclude.
