Since 2015, the United Nations Education, Scientific and Cultural Organization has posited physical literacy as a central tenet and pedagogical reform initiative of quality physical education. An assessment of physical literacy is, therefore, becoming an increasingly important aspect of physical education in schools. Internal and external threats to assessment instrument validity are always important considerations in assessment tool development, but this paper identifies why particular attention needs to be paid to the content validity of one such instrument being used by physical educators and researchers. Specifically, this paper interrogates the content validity claims of the Canadian Assessment of Physical Literacy--Version 2 and discusses why greater attention needs to be paid to the content validity of this instrument if pedagogical reform in physical education is not to be undermined.