Prediction is one characteristic of the human mind. But what does it mean to say the mind is a "prediction machine" and "inherently forward looking" as is frequently claimed? In natural languages, many contexts are not easily predictable in a forward fashion. In English, for example, many frequent verbs do not carry unique meaning on their own but instead, rely on another word or words that follow them to become meaningful. Upon reading "take a" the processor often cannot easily predict "walk" as the next word. But the system can "look back" and integrate "walk" more easily when it follows "take" a (e.g., as opposed to *"make"|"get"|"have a walk"). In the present paper, we provide further evidence for the importance of both forward and backward-looking in language processing. In two self-paced reading tasks and an eye-tracking reading task, we found evidence that adult English native speakers' sensitivity to word forward and backward conditional probability significantly predicted reading times over and above psycholinguistic predictors of reading latencies. We conclude that both forward and backward-looking (prediction and integration) appear to be important characteristics of language processing. Our results thus suggest that it makes just as much sense to call the mind an "integration machine" which is inherently backward 'looking.'