| |
| |
| |
| Toward a Calculus of the Mind/Brain: An Overview | |
| |
| |
| |
| Harmony Optimization and the Computational Architecture of the Mind/Brain | |
| |
| |
| |
| |
| Symbols and neurons: What type of computer is the mind/brain? | |
| |
| |
| |
| Connectionist computation | |
| |
| |
| |
| Symbolic computation and the combinatorial strategy | |
| |
| |
| |
| The combinatorial strategy illustrated: Sentence processing | |
| |
| |
| |
| Other constituent relations | |
| |
| |
| |
| Grammatical and lexical knowledge; argument structure | |
| |
| |
| |
| Explaining the productivity of cognition | |
| |
| |
| |
| The proposed solution | |
| |
| |
| |
| The evidence | |
| |
| |
| |
| Prior evidence | |
| |
| |
| |
| Novel evidence | |
| |
| |
| |
| Strict domination | |
| |
| |
| |
| Universals | |
| |
| |
| |
| Optimality Theory and the cognitive science of language | |
| |
| |
| |
| Philosophical arguments | |
| |
| |
| |
| Recapitulation | |
| |
| |
| |
| Overview of the book | |
| |
| |
| |
| Structure of the book | |
| |
| |
| |
| Chapter-by-chapter summary of the book | |
| |
| |
| References | |
| |
| |
| |
| Principles of the Integrated Connectionist/Symbolic Cognitive Architecture | |
| |
| |
| |
| |
| Symbolic structures as patterns of activation | |
| |
| |
| |
| Symbolic operations as patterns of connections | |
| |
| |
| |
| Optimization in neural networks | |
| |
| |
| |
| Optimization in grammar I: Harmonic Grammar | |
| |
| |
| |
| Optimization in grammar II: Optimality Theory | |
| |
| |
| |
| Universal Grammar, universal grammar, and innate knowledge | |
| |
| |
| |
| Treatment of ICS principles in the book | |
| |
| |
| |
| The big picture: An ICS map | |
| |
| |
| |
| Representations: Form | |
| |
| |
| |
| Representations: Well-formedness; computational perspective | |
| |
| |
| |
| Well-formedness: Empirical perspective | |
| |
| |
| |
| Functions | |
| |
| |
| |
| Processing | |
| |
| |
| |
| Putting it all together | |
| |
| |
| |
| Learning | |
| |
| |
| References | |
| |
| |
| |
| Foundational Implications of the ICS Architecture: Unification in Cognitive Science | |
| |
| |
| |
| |
| Implications for the foundations of cognitive science | |
| |
| |
| |
| The multiple challenges of a unified cognitive science | |
| |
| |
| |
| Substantive integration and the central paradox of cognition | |
| |
| |
| |
| Methodological integration and sectarian warfare | |
| |
| |
| |
| Metatheoretic integration: Model- versus principle-centered research | |
| |
| |
| |
| Connectionism and integration | |
| |
| |
| |
| Illustrative results and integrative goals | |
| |
| |
| |
| Relevance to integrative goals | |
| |
| |
| |
| Caveats | |
| |
| |
| References | |
| |
| |
| |
| Optimality: From Neural Networks to Universal Grammar | |
| |
| |
| |
| |
| Optimality Theory | |
| |
| |
| |
| Markedness and faithfulness constraints | |
| |
| |
| |
| Optimality Theory and neural network theory | |
| |
| |
| |
| Linguistic knowledge and its use | |
| |
| |
| References | |
| |
| |
| |
| Principles of the Integrated Connectionist/Symbolic Cognitive Architecture | |
| |
| |
| |
| Formalizing the Principles I: Representation and Processing in the Mind/Brain | |
| |
| |
| |
| |
| P[subscript 1]: Rep[subscript ICS]-Symbolic structures as patterns of activity | |
| |
| |
| |
| Constituent combination by superposition | |
| |
| |
| |
| Variable binding via the tensor product | |
| |
| |
| |
| Embedding via recursively defined role vectors | |
| |
| |
| |
| The principle Rep[subscript ICS] formalized | |
| |
| |
| |
| P[subscript 2]: Proc[subscript ICS]-Symbolic operations as patterns of connections | |
| |
| |
| |
| Linear processing in neural networks | |
| |
| |
| |
| Symbolic operations in ICS networks | |
| |
| |
| |
| Summary | |
| |
| |
| |
| Multilevel descriptions in ICS | |
| |
| |
| References | |
| |
| |
| |
| Formalizing the Principles II: Optimization and Grammar | |
| |
| |
| |
| |
| Optimization in neural networks | |
| |
| |
| |
| Harmony of an activation vector | |
| |
| |
| |
| Harmony maximization | |
| |
| |
| |
| Optimization in grammar I: Harmonic Grammar | |
| |
| |
| |
| Derivation of the Harmonic Grammar formalism | |
| |
| |
| |
| Relations between lower and higher levels in Harmonic Grammar | |
| |
| |
| |
| Harmonic Grammar and formal languages | |
| |
| |
| |
| A competence/performance distinction | |
| |
| |
| |
| An application to natural language syntax/semantics | |
| |
| |
| |
| Optimization in grammar II: Optimality Theory | |
| |
| |
| References | |
| |
| |
| |
| Symbolic Computation with Activation Patterns | |
| |
| |
| |
| |
| Realizing symbolic structure in activation vectors | |
| |
| |
| |
| The typology | |
| |
| |
| |
| Parietal representation of spatial location | |
| |
| |
| |
| Contextual roles: Propositional memory | |
| |
| |
| |
| Patterns in space-time: Synchronous-firing variable binding | |
| |
| |
| |
| The variable binding problem reviewed | |
| |
| |
| |
| Binding by synchronized firing | |
| |
| |
| |
| Tensor product representations | |
| |
| |
| |
| Synchronized firing as a tensor product | |
| |
| |
| |
| A two-way formal/implementational representation typology | |
| |
| |
| |
| Conclusion | |
| |
| |
| |
| Contraction: Holographic Reduced Representations | |
| |
| |
| |
| Tensor contraction | |
| |
| |
| |
| Holographic Reduced Representations | |
| |
| |
| |
| Squashing: Recursive Auto-Associative Memories | |
| |
| |
| References | |
| |
| |
| |
| Tensor Product Representations: Formal Foundations | |
| |
| |
| |
| |
| Introduction | |
| |
| |
| |
| The problem | |
| |
| |
| |
| Distributed representation and connectionist variable binding | |
| |
| |
| |
| Connectionist representation and tensor product binding: Definition and examples | |
| |
| |
| |
| Connectionist representation | |
| |
| |
| |
| Tensor product representation: Definition | |
| |
| |
| |
| Previous representations and special cases of tensor product realization | |
| |
| |
| |
| Relations among purely local, semilocal, and fully distributed realizations | |
| |
| |
| |
| Tensor product representation: Properties | |
| |
| |
| |
| Unbinding | |
| |
| |
| |
| Graceful saturation | |
| |
| |
| |
| Storage of structured data in connectionist memories | |
| |
| |
| |
| Learning optimal role representations | |
| |
| |
| |
| Extension to recursive structures: Trees | |
| |
| |
| |
| Realizing recursive functions in feed-forward networks | |
| |
| |
| |
| Feedback networks: Grammars | |
| |
| |
| |
| TPPL: Tensor Product Programming Language | |
| |
| |
| |
| Fully distributed recursive representations | |
| |
| |
| References | |
| |
| |
| |
| Optimization in Neural Networks: Harmony Maximization | |
| |
| |
| |
| |
| Groundwork | |
| |
| |
| |
| Discrete-update and continuous-update networks | |
| |
| |
| |
| Feed-forward and symmetric connectivity architectures | |
| |
| |
| |
| Harmony maximization in static and dynamic networks | |
| |
| |
| |
| Quasi-linear units | |
| |
| |
| |
| Thresholds and the bias unit B | |
| |
| |
| |
| Expansion of the core Harmony function | |
| |
| |
| |
| Deterministic neural networks as dynamical systems | |
| |
| |
| |
| Boolean networks | |
| |
| |
| |
| Continuous-valued networks | |
| |
| |
| |
| Summary | |
| |
| |
| |
| Extensions: Beyond Harmony maximization | |
| |
| |
| |
| Stochastic neural networks as probabilistic mental models | |
| |
| |
| |
| Stochastic global Harmony maximization: Boltzmann networks | |
| |
| |
| |
| Modeling a stochastic environment: Harmonium networks | |
| |
| |
| |
| Conclusion | |
| |
| |
| References | |
| |
| |
| |
| Harmonic Grammars and Harmonic Parsers for Formal Languages | |
| |
| |
| |
| |
| Context-free harmonic grammars | |
| |
| |
| |
| Parsing | |
| |
| |
| |
| Grammar preprocessing | |
| |
| |
| |
| Example | |
| |
| |
| |
| Hopfield network | |
| |
| |
| |
| Example | |
| |
| |
| |
| Concluding remarks | |
| |
| |
| References | |
| |
| |
| |
| The Interaction of Syntax and Semantics: A Harmonic Grammar Account of Split Intransitivity | |
| |
| |
| |
| |
| The problem | |
| |
| |
| |
| Split intransitivity crosslinguistically | |
| |
| |
| |
| Split intransitivity in French | |
| |
| |
| |
| A Harmonic Grammar account | |
| |
| |
| |
| Review of Harmonic Grammar | |
| |
| |
| |
| The constraints | |
| |
| |
| |
| The connectionist realization | |
| |
| |
| |
| The networks | |
| |
| |
| |
| Methodological summary | |
| |
| |
| |
| Analyzing the account | |
| |
| |
| |
| Empirical adequacy | |
| |
| |
| |
| Consistency | |
| |
| |
| |
| Comprehensibility | |
| |
| |
| |
| Necessity of semantic and syntactic features | |
| |
| |
| |
| Generality | |
| |
| |
| |
| Extending the account | |
| |
| |
| |
| Conclusion | |
| |
| |
| References | |
| |
| |
| |
| Optimality Theory: The Structure, Use, and Acquisition of Grammatical Knowledge | |
| |
| |
| |
| |
| Principles of Optimality Theory: OT in theoretical linguistics | |
| |
| |
| |
| Structural descriptions | |
| |
| |
| |
| Grammars specify functions; candidates from Gen and Int | |
| |
| |
| |
| Harmonic ordering: H-eval and competition | |
| |
| |
| |
| Violable constraints: Con, Markedness, and Faithfulness | |
| |
| |
| |
| Optimality, harmonic ordering, and constraint ranking | |
| |
| |
| |
| Universality, factorial typology, and Richness of the Base | |
| |
| |
| |
| Lexicon optimization | |
| |
| |
| |
| The structure of Con | |
| |
| |
| |
| Optimality Theory's contributions to linguistic theory | |
| |
| |
| |
| The grammar-processing problem | |
| |
| |
| |
| The grammar-learning problem | |
| |
| |
| |
| Frequently asked "questions": Explanation in Optimality Theory | |
| |
| |
| |
| Conclusion | |
| |
| |
| References | |
| |
| |
| Index | |