In this study, I investigated the simultaneous effects of the reader, the text, and the task factors, and their interactions, on reading comprehension, using explanatory item response models. Analyses of a large data set from a commercially available online assessment system with a wide range of readers (n = 10,547) and passages (n = 48) uncovered factors that contribute to reading challenge in complex ways. Among the passage features, sentence length, word frequency, syntactic simplicity, and temporality were found to significantly affect comprehension difficulty. More importantly, these textual features were moderated by student general vocabulary and task type. In general, high-vocabulary readers benefited more from traditional textual affordances (e.g., shorter sentences, familiar words, simpler grammatical constructions) than low-vocabulary readers, especially when asked to recall localized information without accessing the passage. However, a reverse effect was found with temporality: Passages with more time markers helped low-vocabulary readers, whereas low-temporality passages helped high-vocabulary readers. Ultimately, understanding these complex interactions, as highlighted in the RAND Reading Study Group's heuristic model, will be key in supporting students with their comprehension development.