The usefulness of studying thought processes in the construct validation of ability tests was examined using a sample consisting of 343 Canadian senior high school students. Four levels of probing were used by the interviewer to examine the students' thinking processes while taking the Test on Appraising Observations: (1) think aloud; (2) immediate recall--examinees were asked why they chose that answer; (3) criteria probe--examinees were asked whether a feature of a test item determined the answer; and, (4) principle probe--examinees were also asked whether answer choice was based on particular general principles. Two scores were derived: performance scores or number right, and thinking scores indicating the quality of thinking displayed. Analyses were concerned with three questions: (1) whether the verbal reports accurately portray the thinking that takes place; (2) whether thinking concurrent with reporting differs from thinking in testing situations in which reports are not elicited; and (3) whether thinking subsequent to reporting is different from thinking subsequent to non-reporting testing situations. Analyses of variance were conducted to determine the effects and interactions of interview group, interviewer, sex, and grade level. Seven types of verbal activity were noted during the interviews. (GDC)