Free Access
Harmony in Linguistic Cognition
Paul Smolensky,
Corresponding Author
Paul Smolensky
Cognitive Science Department, Johns Hopkins University
Cognitive Science Department, Johns Hopkins University, Baltimore, MD 21218-2685. E-mail: smolensky@jhu.eduSearch for more papers by this authorPaul Smolensky,
Corresponding Author
Paul Smolensky
Cognitive Science Department, Johns Hopkins University
Cognitive Science Department, Johns Hopkins University, Baltimore, MD 21218-2685. E-mail: smolensky@jhu.eduSearch for more papers by this authorFirst published: 01 September 2006
Citations: 21
Abstract
In this article, I survey the integrated connectionist/symbolic (ICS) cognitive architecture in which higher cognition must be formally characterized on two levels of description. At the microlevel, parallel distributed processing (PDP) characterizes mental processing; this PDP system has special organization in virtue of which it can be characterized at the macrolevel as a kind of symbolic computational system. The symbolic system inherits certain properties from its PDP substrate; the symbolic functions computed constitute optimization of a well-formedness measure called Harmony. The most important outgrowth of the ICS research program is optimality theory (Prince & Smolensky, 1993/2004), an optimization-based grammatical theory that provides a formal theory of cross-linguistic typology. Linguistically, Harmony maximization corresponds to minimization of markedness or structural ill-formedness. Cognitive explanation in ICS requires the collaboration of symbolic and connectionist principles. ICS is developed in detail in Smolensky and Legendre (2006a); this article is a précis of and guide to those volumes.
References
- Anderson, J. A., Silverstein, J. W., Ritz, S. A., & Jones, R. S. (1977). Distinctive features, categorical perception, and probability learning: Some applications of a neural model. Psychological Review, 84, 413–451.
- Anderson, J. R., & Lebiere, C. J. (1998). The atomic components of thought. Mahwah , NJ : Lawrence Erlbaum Associates, Inc.
- Archangeli, D. (1997). Optimality theory: An introduction to linguistics in the 1990s. In D. Archangeli & D. T. Langendoen (Eds.), Optimality theory: An overview (pp. 1–32). Malden , MA : Blackwell.
- Battistella, E. L. (1990). Markedness: The evaluative superstructure of language. Albany : State University of New York Press.
- R. Blutner, & H. Zeevat (Eds.). (2003). Pragmatics in optimality theory. London : Palgrave Macmillan.
-
Bresnan, J. (2000). Optimal syntax. In
J. Dekkers,
F. Leeuw, &
J. Weijer (Eds.), Optimality theory: Phonology, syntax and acquisition (pp. 334–385).
Oxford
,
England
: Oxford University Press.
10.1093/oso/9780198238430.003.0011 Google Scholar
-
Burzio, L. (1994). Principles of English stress.
Cambridge
,
England
: Cambridge University Press.
10.1017/CBO9780511519741 Google Scholar
- M. H. Christiansen, & N. Chater (Eds.). (1999). Special issue on connectionist models of human language processing [Special issue]. Cognitive Science, 23.
- Cohen, M. A., & Grossberg, S. (1983). Absolute stability of global pattern formation and parallel memory storage by competitive neural networks. IEEE Transactions on Systems, Man, and Cybernetics, 13, 815–825.
-
Cummins, R., &
Schwarz, G. (1991). Connectionism, computation, and cognition. In
T. E. Horgan &
J. Tienson (Eds.), Connectionism and the philosophy of mind (pp. 60–73).
Dordrecht
,
The Netherlands
: Kluwer.
10.1007/978-94-011-3524-5_3 Google Scholar
- Davidson, L., Jusczyk, P. W., & Smolensky, P. (2006). Optimality in language acquisition: I. The initial and final states of the phonological grammar. In P. Smolensky & G. Legendre (Eds.), The harmonic mind: From neural computation to optimality-theoretic grammar (Vol. 2, pp. 793–839). Cambridge , MA : MIT Press.
- Dolan, C. P. (1989). Tensor manipulation networks: Connectionist and symbolic approaches to comprehension, learning, and planning. Unpublished doctoral dissertation, University of California, Los Angeles.
- Fodor, J. A. (1997). Connectionism and the problem of systematicity (continued): Why Smolensky's solution still doesn't work. Cognition, 62, 109–119.
- Fodor, J. A., & McLaughlin, B.P. (1990). Connectionism and the problem of systematicity: Why Smolensky's solution doesn't work. Cognition, 35, 183–204.
- Fodor, J. A., & Pylyshyn, Z. W. (1988). Connectionism and cognitive architecture: A critical analysis. Cognition, 28, 3–71.
- Golden, R. M. (1986). The “brain-state-in-a-box” neural model is a gradient descent algorithm. Mathematical Psychology, 30–31, 73–80.
- Golden, R. M. (1988). A unified framework for connectionist systems. Biological Cybernetics, 59, 109–120.
- R. L. Goldstone (Ed.). (2004). 2003 Rumelhart prize special issue honoring Aravind Joshi [Special issue]. Cognitive Science, 28(5).
- R. L. Goldstone (Ed.). (2005). 2004 Rumelhart prize special issue honoring John R. Anderson [Special issue]. Cognitive Science, 29(3).
- Greenberg, J. (1978). Some generalizations concerning initial and final consonant clusters. In J. Greenberg (Ed.), Universals of human language: Vol. 2. Phonology (pp. 243–279). Stanford , CA : Stanford University Press.
- Grimshaw, J. (1997). Projection, heads, and optimality. Linguistic Inquiry, 28, 373–422.
-
Grossberg, S. (1982). Studies of mind and brain: Neural principles of learning, perception, development, cognition, and motor control.
Boston
: Reidel.
10.1007/978-94-009-7758-7 Google Scholar
- Hale, J., & Smolensky, P. (2006). Harmonic grammars and harmonic parsers for formal languages. In P. Smolensky & G. Legendre (Eds.), The harmonic mind: From neural computation to optimality-theoretic grammar (Vol. 1, pp. 393–415). Cambridge , MA : MIT Press.
- Halford, G. S., Wilson, W. H., & Phillips, S. (1998). Processing capacity defined by relational complexity: Implications for comparative, developmental, and cognitive psychology. Behavioral and Brain Sciences, 21, 803–864.
- P. Hendriks, H. Hoop, & H. Swart (Eds.). (2000). Special issue on the optimization of interpretation [Special issue]. Journal of Semantics, 17(3–4).
- G. E. Hinton, & J. A. Anderson (Eds.). (1981). Parallel models of associative memory. Hillsdale , NJ : Lawrence Erlbaum Associates, Inc.
- Hinton, G. E., & Sejnowski, T. J. (1983). Optimal perceptual inference. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (pp. 448–453). IEEE Computer Society Press.
- Hinton, G. E., & Sejnowski, T. J. (1986). Learning and relearning in Boltzmann machines. In D. E. Rumelhart, J. L. McClelland, & the PDP Research Group, Parallel distributed processing: Explorations in the microstructure of cognition (Vol. 1, pp. 282–317). Cambridge , MA : MIT Press.
- Hopcroft, J. E., & Ullman, J. D. (1979). Introduction to automata theory, languages, and computation. Reading , MA : Addison-Wesley.
- Hopfield, J. J. (1982). Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences USA, 79, 2554–2558.
- Hopfield, J. J. (1984). Neurons with graded response have collective computational properties like those of two-state neurons. Proceedings of the National Academy of Sciences USA, 81, 3088–3092.
- Hopfield, J. J. (1987). Learning algorithms and probability distributions in feed-forward and feed-back networks. Proceedings of the National Academy of Sciences USA, 84, 8429–8433.
- Jakobson, R. (1962). Selected writings: I. Phonological studies. The Hague , Netherlands : Mouton.
- Jordan, M. I. (1986). Attractor dynamics and parallelism in a connectionist sequential machine. In Proceedings of the Cognitive Science Society (Vol. 8, pp. 10–17). Hillsdale , NJ : Lawrence Erlbaum Associates, Inc.
- Jurafsky, D. S. (1996). A probabilistic model of lexical and syntactic access and disambiguation. Cognitive Science, 20, 137–194.
-
Kager, R. (1999). Optimality theory.
Cambridge
,
England
: Cambridge University Press.
10.1017/CBO9780511812408 Google Scholar
-
R. Kager,
J. Pater, &
W. Zonneveld (Eds.). (2004). Constraints in phonological acquisition.
Cambridge
,
England
: Cambridge University Press.
10.1017/CBO9780511486418 Google Scholar
-
Kohonen, T. (1977). Associative memory: A system-theoretical approach.
New York
: Springer.
10.1007/978-3-642-96384-1 Google Scholar
-
G. Legendre,
J. Grimshaw, &
S. Vikner (Eds.). (2001). Optimality-theoretic syntax.
Cambridge
,
MA
: MIT Press.
10.7551/mitpress/5161.001.0001 Google Scholar
- Legendre, G., Miyata, Y., & Smolensky, P. (1990). Harmonic Grammar—A formal multi-level connectionist theory of linguistic well-formedness: Theoretical foundations. In Proceedings of the Cognitive Science Society (Vol. 12, pp. 388–395). Lawrence Erlbaum Associates, Inc.
- Legendre, G., Miyata, Y., & Smolensky, P. (2006). The interaction of syntax and semantics: A harmonic grammar account of split intransitivity. In P. Smolensky & G. Legendre, The harmonic mind: From neural computation to optimality-theoretic grammar (Vol. 1, pp. 417–452). Cambridge , MA : MIT Press.
- Legendre, G., Raymond, W., & Smolensky, P. (1993). An optimality-theoretic typology of case and grammatical voice systems. In Proceedings of the Berkeley Linguistics Society (Vol. 19, pp. 464–478). Berkeley , CA : Berkeley Linguistics Society.
- Legendre, G., Raymond, W., & Smolensky, P. (2006). Optimality in syntax: II. Wh-questions. In P. Smolensky & G. Legendre, The harmonic mind: From neural computation to optimality-theoretic grammar (Vol. 2, pp. 183–230). Cambridge , MA : MIT Press.
- Legendre, G., Sorace, A., & Smolensky, P. (2006). The optimality theory-harmonic grammar connection. In P. Smolensky & G. Legendre, The harmonic mind: From neural computation to optimality-theoretic grammar (Vol. 2, pp. 339–402). Cambridge , MA : MIT Press.
- Manning, C. D., & Schütze, H. (1999). Foundations of statistical natural language processing. Cambridge , MA : MIT Press.
- McCarthy, J. J. (2002). A thematic guide to optimality theory. Cambridge , England : Cambridge University Press.
- McCarthy, J. J., & Prince, A. (1995). Faithfulness and reduplicative identity. In J. Beckman, L. Walsh Dickey, & S. Urbanczyk (Eds.), University of Massachusetts occasional papers in linguistics: 18. Papers in optimality theory (pp. 249–384). Amherst : University of Massachusetts at Amherst, GLSA.
- McClelland, J. L. (1979). On the time-relations of mental processes: An examination of systems of processes in cascade. Psychological Review, 86, 287–330.
- McClelland, J. L., Rumelhart, D. E., & the PDP Research Group. (1986). Parallel distributed processing: Explorations in the microstructure of cognition (Vol. 2). Cambridge , MA : MIT Press.
- Minsky, M., & Papert, S. (1969). Perceptrons. Cambridge , MA : MIT Press.
- Misker, J. M. V., & Anderson, J. R. (2003). Combining optimality theory and a cognitive architecture. In Proceedings of the International Conference on Cognitive Modeling (Vol. 5).
- Newell, A. (1990). Unified theories of cognition. Cambridge , MA : Harvard University Press.
- Pinker, S., & Prince, A. (1988). On language and connectionism: Analysis of a parallel distributed processing model of language acquisition. Cognition, 28, 73–193.
- Plate, T. A. (1991). Holographic reduced representations: Convolution algebra for compositional distributed representations. In J. Mylopoulos & R. Reiter (Eds.), Proceedings of the International Joint Conference on Artificial Intelligence (Vol. 12, pp. 30–35). San Mateo , CA : Morgan Kaufmann.
- Plate, T. A. (1994). Distributed representations and nested compositional structure. Unpublished doctoral dissertation, University of Toronto, Toronto, Ontario, Canada.
- Plate, T. A. (2003). Holographic reduced representation: Distributed representation for cognitive structures. Stanford , CA : CSLI Publications.
- Pollack, J. (1988). Recursive auto-associative memory: Devising compositional distributed representations. In Proceedings of the Cognitive Science Society (Vol. 10, pp. 33–39). Hillsdale , NJ : Lawrence Erlbaum Associates, Inc.
- Pollack, J. (1990). Recursive distributed representations. Artificial Intelligence, 46, 77–105.
- Pouget, A., & Sejnowski, T. J. (1997). Spatial transformations in the parietal cortex using basis functions. Journal of Cognitive Neuroscience, 9, 222–237.
- Prince, A., & Smolensky, P. (1991). Notes on connectionism and harmony theory in linguistics (Technical Rep. No. CU-CS-533-91). Boulder : Computer Science Department, University of Colorado at Boulder.
- Prince, A., & Smolensky, P. (19932004). Optimality theory: Constraint interaction in generative grammar. Cambridge , MA : Technical report, Rutgers University and University of Colorado at Boulder, 1993. Rutgers Optimality Archive 537, 2002. Revised version published by Blackwell, 2004.
- Prince, A., & Smolensky, P. (1997). Optimality: From neural networks to universal grammar. Science, 275, 1604–1610.
- Prince, A., & Smolensky, P. (2006). Optimality in phonology: I. Syllable structure. In P. Smolensky & G. Legendre (Eds.), The harmonic mind: From neural computation to optimality-theoretic grammar (Vol. 2, pp. 3–25). Cambridge , MA : MIT Press.
- Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning internal representations by error propagation. In D. E. Rumelhart, J. L. McClelland, & the PDP Research Group, Parallel distributed processing: Explorations in the microstructure of cognition (Vol. 1, pp. 318–362). Cambridge , MA : MIT Press.
-
Rumelhart, D.E., McClelland, J. L., & the PDP Research Group. (1986). Parallel distributed processing: Explorations in the microstructure of cognition (Vol. 1).
Cambridge
,
MA
: MIT Press.
10.7551/mitpress/5236.001.0001 Google Scholar
- Rutgers Optimality Archive. Retrieved from http:roa.rutgers.edu.
- Seidenberg, M. S., & MacDonald, M. C. (1999). A probabilistic constraints approach to language acquisition and processing. Cognitive Science, 23, 568–588.
- Shastri, L., & Ajjanagadde, V. (1993). From simple associations to systematic reasoning: A connectionist representation of rules, variables and dynamic bindings using temporal synchrony. Behavioral and Brain Sciences, 16, 417–494.
- Smolensky, P. (1983). Schema selection and stochastic inference in modular environments. In Proceedings of the National Conference on Artificial Intelligence (Vol. 3, pp. 378–382). San Mateo , CA : William Kaufmann.
- Smolensky, P. (1986). Information processing in dynamical systems: Foundations of harmony theory. In D. E. Rumelhart, J. L. McClelland, & the PDP Research Group (Eds.), Parallel distributed processing: Explorations in the microstructure of cognition (Vol. 1, pp. 194–281). Cambridge , MA : MIT Press.
- Smolensky, P. (1988). On the proper treatment of connectionism. Behavioral and Brain Sciences, 11, 1–74.
- Smolensky, P. (1990). Tensor product variable binding and the representation of symbolic structures in connectionist networks. Artificial Intelligence, 46, 159–216.
- Smolensky, P. (1993). Harmonic grammars for formal languages. In S. Hanson, J. D. Cowan, & C. L. Giles (Eds.), Advances in neural information processing systems (Vol. 5, pp. 847–854). San Mateo , CA : Morgan Kaufmann.
- Smolensky, P. (1995). Constituent structure and explanation in an integrated connectionist/symbolic cognitive architecture. In C. Macdonald & G. Macdonald (Eds.), Connectionism: Debates on psychological explanation (Vol. 2, pp. 221–290). Oxford , England : Blackwell.
- Smolensky, P. (2006a). Computational levels and integrated connectionist/symbolic explanation. In P. Smolensky & G. Legendre, The harmonic mind: From neural computation to optimality-theoretic grammar (Vol. 2, pp. 503–592). Cambridge , MA : MIT Press.
- Smolensky, P. (2006b). Formalizing the principles: I. Representation and processing in the mind/brain. In P. Smolensky & G. Legendre, The harmonic mind: From neural computation to optimality-theoretic grammar (Vol. 1, pp. 147–205). Cambridge , MA : MIT Press.
- Smolensky, P. (2006c). Formalizing the principles: II. Optimization and grammar. In P. Smolensky & G. Legendre, The harmonic mind: From neural computation to optimality-theoretic grammar (Vol. 1, pp. 207–234). Cambridge , MA : MIT Press.
- Smolensky, P. (2006d). Optimality in phonology: II. Harmonic completeness, local constraint conjunction, and feature domain markedness. In P. Smolensky & G. Legendre, The harmonic mind: From neural computation to optimality-theoretic grammar (Vol. 2, pp. 27–160). Cambridge , MA : MIT Press.
- Smolensky, P. (2006e). Optimization in neural networks: Harmony maximization. In P. Smolensky & G. Legendre, The harmonic mind: From neural computation to optimality-theoretic grammar (Vol. 1, pp. 345–392). Cambridge , MA : MIT Press.
- Smolensky, P. (2006f). Tensor product representations: Formal foundations. In P. Smolensky & G. Legendre (Eds.), The harmonic mind: From neural computation to optimality-theoretic grammar (Vol. 1, pp. 271–344). Cambridge , MA : MIT Press.
- Smolensky, P., & Legendre, G. (2006a). The harmonic mind: From neural computation to optimality-theoretic grammar: Vol. 1. Cognitive architecture. Vol 2: Linguistic and philosophical implications. Cambridge , MA : MIT Press.
- Smolensky, P., & Legendre, G. (2006b). Principles of the integrated connectionist/symbolic cognitive architecture. In P. Smolensky & G. Legendre, The harmonic mind: From neural computation to optimality-theoretic grammar (Vol. 1, pp. 63–97). Cambridge , MA : MIT Press.
- Smolensky, P., Legendre, G., & Tesar, B. B. (2006). Optimality theory: The structure, use and acquisition of grammatical knowledge. In P. Smolensky & G. Legendre (Eds.), The harmonic mind: From neural computation to optimality-theoretic grammar (Vol. 1, pp. 453–544). Cambridge , MA : MIT Press.
- Smolensky, P., & Tesar, B. B. (2006). Symbolic computation with activation patterns. In P. Smolensky & G. Legendre, The harmonic mind: From neural computation to optimality-theoretic grammar (Vol. 1, pp. 235–270). Cambridge , MA : MIT Press.
- Soderstrom, M., Mathis, D. W., & Smolensky, P. (2006). Abstract genomic encoding of universal grammar in Optimality Theory. In P. Smolensky & G. Legendre, The harmonic mind: From neural computation to optimality-theoretic grammar (Vol. 2, pp. 403–471). Cambridge , MA : MIT Press.
- Stemberger, J. P., & Bernhardt, B. H. (1998). Handbook of phonological development from the perspective of constraint-based nonlinear phonology. San Diego , CA : Academic.
- Stevenson, S., & Smolensky, P. (2006). Optimality in sentence processing. In P. Smolensky & G. Legendre, The harmonic mind: From neural computation to optimality-theoretic grammar (Vol. 2, pp. 307–338). Cambridge , MA : MIT Press.
- Tabor, W. (2000). Fractal encoding of context-free grammars in connectionist networks. Expert Systems, 17, 41–56.
- Tesar, B. B., & Smolensky, P. (1994). Synchronous-firing variable binding is spatio-temporal tensor product representation. In Proceedings of the Cognitive Science Society (Vol. 16, pp. 870–875). Hillsdale , NJ : Lawrence Erlbaum Associates, Inc.
- Wilson, C. (2003). Experimental investigation of phonological naturalness. In G. Garding & M. Tsujimura (Eds.), Proceedings of the West Coast Conference on Formal Linguistics (Vol. 22, pp. 533–546). Somerville , MA : Cascadilla.
