Although the recent identification of the five critical components of early literacy has been a catalyst for modifications to the content of materials used to provide reading instruction and the tools used to examine student's acquisition of early literacy skills, these skills have not received equal attention from test developers and publishers. In particular, a review of early literacy available measures for screening and monitoring students reveals a dearth of tools for examining different facets of reading comprehension. The purposes of this study were twofold: (a) to examine the relative difficulty of items written to assess literal, inferential, and evaluative comprehension, and (b) to compare single factor and bifactor models of reading comprehension to determine if items written to assess students' literal, inferential, and evaluative comprehension abilities comprise unique measurement factors. Data from approximately 2,400 fifth grade students collected in the fall, winter, and spring of fifth grader were used to examine these questions. Findings indicated that (a) the relative difficulty of item types may be curvilinear, with literal items being significantly less challenging than inferential and evaluative items, and (b) literal, inferential, and evaluative comprehension measurement factors explained unique portions of variance in addition to a general reading comprehension factor. Instructional implications of the findings are discussed.