Psychometrics is the field of designing tests and assessments to measure certain psychological concepts. It is chiefly concerned with two fundamental properties: reliability and validity. These properties are often influenced by confounding variables: other things that can influence performance but are not what you are trying to measure. Here, I describe how teaching an undergraduate course on psychometrics has reframed my views of assessments in teaching. The goal of any assessment is to accurately, or validly, measure students' knowledge and learning. This raises three questions that can influence assessment design, course structure, and teaching style: 1) Do deadlines really matter? 2) Should all assessments be offered multiple times? 3) How much do my assessments really measure student knowledge on this topic? In discussing each of these, I consider the ways in which evaluating how validity plays a role in our assessment design and teaching style can help us minimize confounding variables and ensure that students' grades reflect their learning.