This paper describes the benefits of a psychometric analytic approach when studying students' ability to comprehend texts. A reading comprehension measure consisting of four text passages with 32 questions was developed, and analyses using a mean-based approach and linear logistic test (LLTM) method were performed. According to ANOVA, eighth-grade students (n = 160) tend to comprehend texts well with the measure when the texts are familiar and interesting to them and expository. According to the LLTM analysis, readers are likely to comprehend texts well when they are familiar with and interested in the texts, when the texts are narrative, and when the items are text based and nontemporal. This suggests developing comprehension assessments by integrating reader attributes and task attributes in the measurement and analysis of comprehension proficiency.