This study examines the underlying mechanisms of ableism and disablism in the assessment of student learning in higher education. Globally, higher education institutions rely strongly on assessment accommodations (e.g., extra time in tests) to ensure disabled students' participation in assessment. This is also the case in Finland. Even though research on disabled students' experiences of assessment has repeatedly shown that both assessment and assessment accommodations cause barriers for disabled students' inclusion, critically oriented research on this topic has been scarce. In this study, the frameworks of ableism and disablism are used to unveil how assessment is predominantly designed for "the ideal, able student" and how disabled students are framed as "the Other" through assessment. This work is based on an analysis of 139 disabled students' experiences of assessment and assessment accommodations as collected through an open-ended, institution-wide survey at a Finnish university. The findings reveal the profound role of assessment in excluding and marginalizing disabled students as unfit to take part in the testing cultures of academia. The accommodation model is shown to hold disabled people responsible for their own exclusion. Disablism is identified in students' experiences of outright discrimination, such as teachers denying access to assessment accommodations when they are officially granted. This study offers a novel, critical means of discussing assessment from the viewpoints of diversity and inclusion. It also proposes future trajectories for anti-ableist assessment approaches that understand diversity as enriching, rather than obscuring, assessment.