Free Access
A Memory-Based Theory of Verbal Cognition
Simon Dennis,
Corresponding Author
Simon Dennis
Institute of Cognitive Science, University of Colorado
Institute of Cognitive Science, University of Colorado, CO 80301. E-mail: dennissj@psych.colorado.eduSearch for more papers by this authorSimon Dennis,
Corresponding Author
Simon Dennis
Institute of Cognitive Science, University of Colorado
Institute of Cognitive Science, University of Colorado, CO 80301. E-mail: dennissj@psych.colorado.eduSearch for more papers by this authorFirst published: 01 March 2005
Citations: 30
Abstract
The syntagmatic paradigmatic model is a distributed, memory-based account of verbal processing. Built on a Bayesian interpretation of string edit theory, it characterizes the control of verbal cognition as the retrieval of sets of syntagmatic and paradigmatic constraints from sequential and relational long-term memory and the resolution of these constraints in working memory. Lexical information is extracted directly from text using a version of the expectation maximization algorithm. In this article, the model is described and then illustrated on a number of phenomena, including sentence processing, semantic categorization and rating, short-term serial recall, and analogical and logical inference. Subsequently, the model is used to answer questions about a corpus of tennis news articles taken from the Internet. The model's success demonstrates that it is possible to extract propositional information from naturally occurring text without employing a grammar, defining a set of heuristics, or specifying a priori a set of semantic roles.
References
- Allison, L., Powell, D., & Dix, T. I. (1999). Compression and approximate matching. Computer Journal, 42, 1–10.
- Allison, L., Wallace, C. S., & Yee, C. N. (1992). Finite-state models in the alignment of macromolecules. Journal of Molecular Evolution, 35(1), 77–89.
- Anderson, J. R. (1983). The architecture of cognition. Cambridge , MA : Harvard University Press.
- Anderson, J. R. (1993). Rules of the mind. Hillsdale , NJ : Lawrence Erlbaum Associates, Inc.
- Anderson, J. R., & Lebiere, C. (1998). The atomic components of thought. Mahwah , NJ : Lawrence Erlbaum Associates, Inc.
- Anderson, J. R., & Matessa, M. (1997). A production system theory of serial memory. Psychological Review, 104, 728–748.
- Argamon-Engelson, S., Dagan, I., & Krymolowski, Y. (1999). A memory-based approach to learning shallow natural language patterns. Journal of Experimental and Theoretical Artificial Intelligence, 11, 369–390.
- Baird, B., Troyer, T., & Eckman, F. H. (1992, December). Synchronization and grammatical inference in an oscillating Elman net. Paper presented at the Neural Information Processing Systems Conference, Denver, CO.
- Barzilay, R., & Lee, L. (2003, May). Learning to paraphrase: An unsupervised approach using multiple-sequence alignment. Paper presented at the Proceedings of HLT–NAACL, Edmonton, Alberta, Canada.
- Bever, T. G. (1970). The cognitive basis for linguistic structures. In J. R. Hayes (Ed.), Cognition and the development of language (pp. 279–352). New York : Wiley.
- Blaheta, D., & Charniak, E. (2000, May). Assigning function tags to parsed text. Paper presented at the first annual meeting of the North American Chapter of the ACL (NAACL), Seattle, Washington.
- Bod, R. (1992, August). A computational model of language performance: Data oriented parsing. Paper presented at COLING-92, Nantes, France.
- Bransford, J. D., Barclay, J. R., & Franks, J. J. (1972). Sentence memory: A constructive versus interpretative approach. Cognitive Psychology, 3, 193–209.
- Brown, G. D. A., Preece, T., & Hulme, C. (2000). Oscillator-based memory for serial recall. Psychological Review, 107, 127–181.
- Brown, R., & Berko, J. (1960). Word association and the acquisition of grammar. Child Development, 31, 1–14.
- Budiu, R. (2001). The role of background knowledge in sentence processing. Unpublished doctoral dissertation, Carnegie Mellon University, Pittsburgh, PA.
- Burgess, N., & Hitch, G. J. (1992). Towards a network model of the articulatory loop. Journal of Memory and Language, 31, 429–460.
-
Chomsky, N. (1957). Syntactic structures.
The Hague
,
Netherlands
: Mouton.
10.1515/9783112316009 Google Scholar
- Christiansen, M. H., & Chater, N. (1999). Towards a connectionist model of recursion in human linguistic performance. Cognitive Science, 23, 157–206.
- Collins, A., & Loftus, E. (1975). A spreading activation theory of semantic memory. Psychological Review, 82, 407–428.
- Collins, M., & Quillian, M. R. (1969). Retrieval from semantic memory. Journal of Verbal Learning and Verbal Behavior, 8, 240–247.
-
Dempster, A. P.,
Laird, N. M., &
Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm (with discussion).
Journal of the Royal Statistical Society Series B, 39, 1–38.
10.1111/j.2517-6161.1977.tb01600.x Google Scholar
- Dennis, S. (2003a, August). An alignment-based account of serial recall. Paper presented at the 25th annual meeting of the Cognitive Science Society, Boston.
- Dennis, S. (2003b, August). A comparison of statistical models for the extraction of lexical information from text corpora. Paper presented at the 25th annual meeting of the Cognitive Science Society, Boston.
- Dennis, S. (2004). An unsupervised method for the extraction of propositional information from text. Proceedings of the National Academy of Sciences, 101, 5206–5213.
- Dennis, S., & Humphreys, M. S. (2001). A context noise model of episodic word recognition. Psychological Review, 108, 452–478.
- Dress, A., Morgenstern, B., & Stoye, J. (1998). The number of standard and of effective multiple alignments. Applied Mathematics Letters, 11(4), 43–49.
- Elman, J. L. (1991). Distributed representations, simple recurrent networks and grammatical structure. Machine Learning, 7, 195–225.
- Elman, J. L. (1993). Learning and development in neural networks: The importance of starting small. Cognition, 48, 71–99.
- Entwisle, D. R. (1966). Word associations of young children. Baltimore : John Hopkins University Press.
- Erickson, T., & Mattson, M. (1981). From words to meaning: A semantic illusion. Journal of Verbal Learning and Verbal Behavior, 20, 540–552.
- Ericsson, K. A., & Kintsch, W. (1995). Long term working memory. Psychological Review, 102, 211–245.
- Ervin, S. M. (1961). Changes with age in the verbal determinants of word association. American Journal of Psychology, 74, 361–372.
-
Ervin-Tripp, S. M. (1970). Substitution, context and association. In
L. Postman &
G. Keppel (Eds.), Norms of word association (pp. 383–467).
New York
: Academic.
10.1016/B978-0-12-563050-4.50012-1 Google Scholar
- Evans, T. G. (1964). A heuristic program to solve analogy problems. Proceedings of the Spring Joint Computer Conference, 25, 327–338.
- Falkenhainer, B., Forbus, K. D., & Gentner, D. (1989). The structure-mapping engine: Algorithm and examples. Artificial Intelligence, 41, 1–63.
- Fodor, J., Bever, T. G., & Garrett, M. F. (1974). The psychology of language. New York : McGraw-Hill.
- Fodor, J., & Pylyshyn, Z. (1988). Connectionism and cognitive architecture. Cognition, 28, 3–71.
-
French, R. M. (1995). The subtlety of sameness: A theory and computer model of analogy-making.
Cambridge
,
MA
: MIT Press.
10.7551/mitpress/6549.001.0001 Google Scholar
- Gentner, D. (1983). Structure-mapping—A theoretical framework for analogy. Cognitive Science, 7, 155–170.
- Gildea, D., & Jurafsky, D. (2002). Automatic labeling of semantic roles. Computational Linguistics, 28, 245–288.
- Gotoh, O. (1982). An improved algorithm for matching biological sequences. Journal of Molecular Biology, 162, 705–708.
-
Hadley, R. F. (1994). Systematicity in connectionist language learning.
Mind and Language, 9, 247–272.
10.1111/j.1468-0017.1994.tb00225.x Google Scholar
- Halford, G., Wilson, W., Guo, K., Gayler, R., Wiles, J., & Stewart, J. (1994). Connectionist implications for processing capacity limitations in analogies. In K. J. Holyoak & J. Barnden (Eds.), Analogical connections (Vol. 2, pp. 363–415). Norwood , MA : Ablex.
- Harrington, M., & Dennis, S. (2003, August). Structural priming in sentence comprehension. Paper presented at the 25th annual meeting of the Cognitive Science Society, Boston.
- Henson, R. N. A. (1998). Short-term memory for serial order: The start-end model. Cognitive Psychology, 36, 73–137.
- Hitch, G. J., Burgess, N., Towse, J. N., & Culpin, V. (1996). Temporal grouping effects in immediate recall: A working memory analysis. Quarterly Journal of Experimental Psychology Section A–Human Experimental Psychology, 49A, 116–139.
- Hofstadter, D. (1995). Fluid concepts and creative analogies: Computer models of fundamental mechanisms of thought. New York : Basic Books.
- Holyoak, K. J., & Thagard, P. (1989). Analogical mapping by constraint satisfaction. Cognitive Science, 13, 295–355.
- Hummel, J., & Holyoak, K. J. (1997). Distributed representations of structure: A theory of analogical access and mapping. Psychological Review, 104, 427–466.
- Kintsch, W. (1993). Information accretion and reduction in text processing: Inferences. Discourse Processes, 16, 193–202.
- Kintsch, W. (1998). Comprehension: A paradigm for cognition. New York : Cambridge University Press.
- Kokinov, B., & French, R. M. (2003). Computational models of analogy-making. In L. Nadel (Ed.), Encyclopedia of the cognitive sciences (pp. 113–118). New York : Macmillan.
- Kwantes, P. J., & Mewhort, D. J. K. (1999). Modeling lexical decision and word naming as a retrieval process. Canadian Journal of Experimental Psychology, 53, 306–315.
- Landauer, T. K., & Dumais, S. T. (1997). A solution to Plato's problem: The latent semantic analysis theory of the acquisition, induction, and representation of knowledge. Psychological Review, 104, 211–240.
- Levenshtein, V. I. (1965). Binary codes capable of correcting deletions, insertions and reversals. Doklady Akademii Nauk SSSR, 163, 845–848.
- Lewandowsky, S., & Murdock, B. B. (1989). Memory for serial order. Psychological Review, 96, 25–57.
-
Lin, D., &
Pantel, P. (2001). Discovery of inference rules for question answering.
Natural Language Engineering, 7, 343–360.
10.1017/S1351324901002765 Google Scholar
- Lund, K., & Burgess, C. (1996a). Hyperspace analogue to language (HAL): A general model semantic representation. Brain and Cognition, 30(3), 5–5.
- Lund, K., & Burgess, C. (1996b). Producing high-dimensional semantic spaces from lexical co-occurrence. Behavior Research Methods Instruments & Computers, 28, 203–208.
- Marcus, G. F. (1998). Can connectionism save constructivism? Cognition, 66, 153–182.
- Marks, L. E. (1968). Scaling of grammaticalness of self-embedded English sentences. Journal of Verbal Learning and Verbal Behavior, 7, 965–967.
- McNeill, D. (1966). A study of word association. Journal of Verbal Learning and Verbal Behavior, 2, 250–262.
-
Miller, G. A.,
Beckwith, R.,
Fellbaum, C.,
Gross, D., &
Miller, K. J. (1990). Introduction to WordNet: An on-line lexical database.
International Journal of Lexicography, 3
235–244.
10.1093/ijl/3.4.235 Google Scholar
- Miller, G. A., & Isard, S. (1964). Free recall of self-embedded English sentences. Information and Control, 7, 292–303.
- Miller, G. A., & Selfridge, J. A. (1950). Verbal context and the recall of meaningful material. American Journal of Psychology, 63, 176–185.
- Mitchell, M. (1993). Analogy-making as perception: A computer model. Cambridge , MA : MIT Press.
- Narayanan, S., & Jurafsky, D. (1998, July). Bayesian models of human sentence processing. Paper presented at the 20th annual meeting of the Cognitive Science Society, Madison, WI.
- Navarro, D., & Lee, M. (2002, August). Commonalities and distinctions in featural stimulus representations. Paper presented at the 24th annual meeting of the Cognitive Science Society.
- Needleman, S. B., & Wunsch, C. D. (1970). A general method applicable to the search for similarities in the amino acid sequence of two proteins. Journal of Molecular Biology, 48, 443–453.
- Nelson, D. L., McEnvoy, C. L., & Schreiber, T. A. (1999). The University of South Florida word association, rhyme, and word fragment norms. Retrieved January 5, 2005, from http:w3.usf.eduFreeAssociation.
- Newell, A. (1990). The unified theories of cognition: Cambridge , MA : Harvard University Press.
-
Ng, L. H. H., &
Maybery, T. M. (2002). Grouping in verbal short-term memory: Is position coded temporally?
Quarterly Journal of Experimental Psychology, 55A, 391–424.
10.1080/02724980143000343 Google Scholar
- Page, M. P. A., & Norris, D. (1998). The primacy model: A new model of immediate serial recall. Psychological Review, 105, 761–781.
- Palmer, M., Rosenzweig, J., & Cotton, S. (2001, March). Automatic predicate argument analysis of the Penn TreeBank. Paper presented at HLT 2001, First International Conference on Human Language Technology Research, San Diego.
- Petrey, S. (1977). Word association and the development of lexical memory. Cognition, 5, 57–71.
- Phillips, S. (2000). Constituent similarity and systematicity: The limits of first-order connectionism. Connection Science, 12, 45–63.
-
Radford, A. (1988). Transformational grammar: A first course.
Cambridge
,
England
: Cambridge University Press.
10.1017/CBO9780511840425 Google Scholar
- Reder, L., & Kusbit, G. (1991). Locus of the Moses illusion: Imperfect encoding, retrieval or match? Journal of Memory and Language, 30, 385–406.
- Rumelhart, D. E., & Abrahamson, A. A. (1973). A model for analogical reasoning. Cognitive Psychology, 5, 1–28.
- Sankoff, D., & Kruskal, J. B. (1983). Time warps, string edits and macromolecules: The theory and practice of sequence comparison: Reading , MA : Addison-Wesley.
- Scha, R., Bod, R., & Sima'an, K. (1999). A memory-based model of syntactic analysis: Data-oriented parsing. Journal of Experimental and Theoretical Artificial Intelligence, 11, 409–440.
-
Sellers, P. H. (1974). An algorithm for the distance between two finite sequences.
Journal of Combinatorial Theory, 16, 253–258.
10.1016/0097-3165(74)90050-8 Google Scholar
- Shiffrin, R. M., & Steyvers, M. (1997). Model for recognition memory: REM—Retrieving effectively from memory. Psychonomic Bulletin & Review, 4, 145–166.
- Smith, E. E., Shoben, E. J., & Rips, L. J. (1974). Structure and process in semantic memory: A feature model for semantic decision. Psychological Review, 81, 214–241.
- Tversky, A. (1977). Features of similarity. Psychological Review, 84, 327–352.
- van Oostendorp, H., & de Mul, S. (1990). Moses beats Adam: A semantic relatedness effect on a semantic illusion. Acta Psychologica, 74, 35–46.
- van Oostendorp, H., & Kok, I. (1990). Failing to notice errors in sentences. Language and Cognitive Processes, 5, 105–113.
- Waterman, M. S. (1984). General methods of sequence comparison. Bulletin of Mathematical Biology, 46, 473–500.
- Waterman, M. S., Smith, T. F., & Beyer, W. A. (1976). Some biological sequence metrics. Advanced Mathematics, 20, 367–387.
- Wiles, J., & Bloesch, A. (1991). Operators and curried functions: Training and analysis of simple recurrent networks. Paper presented at the Neural Information Processing Systems Conference, San Mateo, CA.
- Wiles, J., & Elman, J. L. (1995, April). Learning to count without a counter: A case study of synamics and activation landscapes in recurrent networks. Paper presented at the annual meeting of the Cognitive Science Society, Brisbane, Australia.
-
Wilson, W.,
Halford, G.,
Gray, B., &
Phillips, S. (2001). The STAR-2 model for mapping hierarchically structured analogs. In
D. Gentner,
K. J. Holyoak, &
B. Kokinov (Eds.), The analogical mind (pp. 125–159).
Cambridge
,
MA
: MIT Press.
10.7551/mitpress/1251.003.0007 Google Scholar
