The use of Likert scale items for measuring teachers' beliefs quantitatively is criticized. In this study, two quantitative methods to measure teachers' beliefs, a Likert scale instrument as well as a new instrument employing rank-then-rate items, are each compared with a qualitative approach at obtaining beliefs, i.e. interviews along with observations. In the first comparison, teachers' responses to the Likert scale instrument seem to not fit the beliefs interpreted from the qualitative approach, particularly regarding beliefs about teaching and learning. Further analyses show that a tendency to respond according to social desirability as well as unspecific teaching contexts seem to be responsible for these inconsistencies. In the second comparison, since the rank-then-rate instrument also takes students' abilities as a teaching context into account, we observed a better fit between teachers' responses to the instrument and the qualitative approach. The results of both comparisons indicate the importance of considering teaching contexts as well as alternatives to Likert scale items because of social desirability in order to have a better prediction about teachers' beliefs.