Students of introductory economics are often able to predict changes in equilibrium price correctly on standardized assessments, but make consistent errors in predicting changes in equilibrium quantity. To examine the reasons for this pattern, the authors collected open-ended explanations written by students and categorized their reasoning using a rigorous multi-step qualitative method. Integrating the qualitative analysis with quantitative data, they find that students exhibit remarkable consistency in their reasoning errors. Common multiple choice assessments tend to reward some types of reasoning errors and thereby make it harder for students to acquire the correct reasoning method. The authors demonstrate that, with thoughtful consideration to avoid excessive subjectivity, a qualitative study can deepen our contextual understanding of the primarily quantitative assessment metrics utilized in economics education research.