Educational reform priorities such as emphasis on quantitative modelling (QM) have positioned undergraduate biology instructors as designers of QM experiences to engage students in authentic science practices that support the development of data-driven and evidence-based reasoning. Yet, little is known about how biology instructors adapt to the pedagogical movement towards incorporating QM opportunities for students in the courses they teach. This study presents the development of the Quantitative Modelling Observation Protocol (QMOP), a classroom observation instrument designed to support the need to characterise various approaches that instructors use to implement QM instruction in undergraduate biology. QMOP provides information about the breadth and depth of QM implementation across three dimensions -- authentic instruction, teaching for understanding, and quantitative approach to teaching biology. We present an interpretive argument, the chain of assumptions we made in relation to the intended use of the instrument, and evidence to assess the validity of our assumptions and inferences about observation scores generated using the instrument. Strengths and weaknesses of evidence pertaining to assumptions about scoring, generalisability, extrapolation, and implications will be discussed to build a validity argument for observations and demonstrate how the instrument can be used for investigating QM instruction in undergraduate biology courses.