We developed a version of AutoTutor that helps struggling adult learners improve their comprehension strategies through conversational agents. We hypothesized that the accuracy and time to answer questions during the conversation could be diagnostic of their mastery of different reading comprehension components: words, textbase, situation model, and rhetorical structure. The results show that adults' performance on more basic reading components (i.e., meaning of words) was higher than on the deeper discourse levels. In contrast, time did not vary significantly among the theoretical levels. The results suggested that adults with low literacy had higher mastery on basic reading levels than deeper discourse levels. The tracking of performance on the four theoretical levels can provide a more nuanced diagnosis of reading problems than a single overall performance score and ultimately improve the adaptivity of an ITS like AutoTutor. [This paper was published in: R. Nkambou, R. Azevedo & J. Vassileva (Eds.), "Proceedings of the 14th International Conference on Intelligent Tutoring Systems" (pp 463-465). Montreal, Cananda: Springer.]