Control Systems for Speech, Language, and Intelligence

Carmen Alonso · April 18, 2025

Control theory is fundamental in the design and understanding of many natural and engineered systems, from cars and robots to power networks and bacterial metabolism. In the context of the brain, one of the most prominent application of control theory is the field of motor control. In this talk, we explore how the principles of control —formalized with control theory— have a much broader set of applications in neuroscience, cognitive science, and intelligent behavior. We focus on language applications, particularly language processing and grounding in technology, as well as speech processing in the human brain. We discuss three applications that exemplify the importance of control across a broad range of settings and research questions. First, we discuss how to leverage language embeddings with control to ground natural language commands in robot actions. We demonstrate how, using these insights, natural language commands can be used to directly instruct a robotic arm to perform a wide range of tasks while preserving safety guarantees. Then, we illustrate how control-theoretic principles can be used to steer the generation of foundation models. We illustrate how, by actively controlling per-layer activations, it is possible to steer a language model away from toxic content, or towards persona-specific responses. We discuss the potential of this work in both enhancing language models during post-training, as well as implications for mechanistic interpretability of embedding spaces in the context of Natural Language Processing. Lastly, we present how control-theoretic models of the brain’s auditory-articulatory system can be used to explain existing experimental results in a unifying framework.