Lecture: The Tensor Brain

A Unified Theory of Perception, Memory and Semantic Decoding

In our proposed model, the cognitive brain state reflects the dynamic interplay between sensory inputs and higher-order cognitive processes. It is continuously shaped by subsymbolic (bottom-up) signals—originating from sensory modalities, internal processing modules, and reward systems—and guided by symbolic (top-down) influences, which encode elements such as recognition, goals, decisions, temporal markers, predicates, and actions. 
 

We posit that the governing principles of bottom-up and top-down interactions can be derived from the mathematical structure of quantum theory. In this framework, a system's state is influenced not only by external inputs, but also by generative measurements initiated by an agent-observer—a notion we describe as Heisenberg-style measurement. The corresponding  probabilistic process mirrors Bayesian inference but diverges in key ways: where traditional Bayesian updates may require approximations (e.g., variational methods), the Heisenberg-style formulation offers a tractable alternative for state updating under uncertainty. 
 

In bottom-up processing, sensory information activates the subsymbolic representation layer, which in turn engages corresponding symbolic structures in the index layer. In top-down processing, symbolic indices activate and modulate the cognitive brain state, which can influence earlier sensory and perceptual stages through embodiment. This mechanism supports semantic memory by integrating abstract, symbolic knowledge with perceptual experience, and serves as the basis for episodic memory, enabling the reconstruction of past perceptual, emotional, reward, and action states. 
 

About Prof. Dr. Tresp:

Volker Tresp is a professor at Ludwig Maximilian University of Munich (LMU). He received his Diploma degree in physics from the University of Göttingen in 1984 and M.Sc., M.Phil. and Ph.D. degrees from Yale University in 1986 and 1989, respectively. During his Ph.D., he worked in Yale’s Image Processing and Analysis Group (IPAG). In 1990, he joined Siemens where he has been heading various research teams in machine learning. In 1997, he became Siemens Inventor of the Year for his innovations in neural networks research  and in 2018 became the first Siemens Distinguished Research Scientist. He revolutionized steel processing by pioneering a novel Bayesian neural network approach that cleverly integrated real-world data with simulated data from a prior solution.  In 1994 he was a visiting scientist at the Massachusetts Institute of Technology in the Center for Biological and Computational Learning, working with the teams of Tomaso Poggio and Michael I. Jordan. He was co-editor of Advances in Neural Information Processing Systems 13.  In 2011, he was appointed professor in informatics at the LMU, where he teaches a course on machine learning and where he is leading a second research team. He is known for his work on Bayesian machine learning, in particular the Bayesian Committee Machine and his work on hierarchical learning with Gaussian processes. The IHRM, the SRM,  SUNS, and RESCAL reflect his work in representation learning for multi-relational graphs. His team has been doing pioneering work on machine learning with knowledge graphs, temporal knowledge graphs, and scene graph analysis.  The work on the Tensor Brain reflects his interest in mathematical models for cognition and neuroscience. In 2020, he became a Fellow of the European Laboratory for Learning and Intelligent Systems (ELLIS). As co-director (with Kristian Kersting and Paolo Frasconi), he leads the ELLIS program “Semantic, Symbolic and Interpretable Machine Learning”.