Research in discourse processing has provided us with a strong foundation for understanding the characteristics of text and discourse, as well as their influence on our processing and representation of texts. However, recent advances in computational techniques have allowed researchers to examine discourse processes in new ways. The purpose of the current paper is to build on prior work in this domain and describe how new methodologies that consider the multi-dimensional nature of texts can serve as a complement to the existing literature. We focus on natural language processing (NLP) methodologies, in which computers calculate information about the linguistic and semantic properties of language data. We first provide a context for the origins of computational discourse analysis through the integration of research across computer science and psychology. We then provide an overview of different NLP methodologies and describe prior work that has leveraged these techniques to advance theoretical perspectives of discourse comprehension and production. Finally, we propose new areas of research that integrate these advances with traditional research methodologies in the field. [This is the online first version of an article published in "Language and Linguistics Compass."]