Course grades are commonly used as an evaluation metric within institutions and as part of education research. However, using grades to compare across course sections may implicitly assume that grades are awarded similarly and consistently. This article details how different sections of the same course offered differing amounts of extra credit and adjusted letter grades to different extents at the end of the term ("post hoc"). In one section, extra credit altered the letter grades of 26.6% of students, and post hoc adjustments altered the letter grades of 35.4% of students. In contrast, in a concurrently-offered section, 1.7% of student grades changed due to extra credit, and 4.3% due to post hoc adjustments. This may complicate some grade-based assessments of instructors, curricula, pedagogical practices, or students. We hope this catalyzes further study into how widespread this phenomenon is, what mechanisms influence it, and what the implications are. Meanwhile, we suggest that education researchers might consider explicitly discussing any available evidence that grades are consistently awarded and/or the possible repercussions of any inconsistency. When not possible, this might be discussed as a study limitation.