Canonical neural networks perform active inference

All right. Welcome. It is ActInf livestream number 51.0. This is the background and context first discussion for Canonical Neural Networks Perform Active Inference by Isomura et al. It's October 26, 2022. Welcome to the Active Inference Institute. We're a participatory online institute that is communicating, learning, and practicing applied active inference. You can find us at some of the links on the page. This is a recorded and an archived livestream, so please provide us with feedback so we can improve our work. All backgrounds and perspectives are welcome and will be following video etiquette for solo livestreams. Head to activeinference.org to learn how to get involved with Active Inference Institute projects. Today, it's the first of several discussions that we'll have on the paper, Canonical Neural Networks Perform Active Inference, 2022, by Takuya Isomura, Hideaki Shimizaki, and Carl Fristen. The video is just an introduction to some of the ideas. It's not a review or final word. There will be an overview of the structure of the paper, and then we'll go through many of the key points. Also, just to disclaim, there's many, many other and better resources to learn about neural networks. So I would very much welcome those with a technical understanding of neural networks and or some of the more applied computational or otherwise aspects of neural networks. It would be awesome to have them on for the dot one and dot two because it was not an area I was familiar with. And so hope that I can hear more from the authors in our coming weeks and others. I'm Daniel. I'm a researcher in California, and this will just be a solo dot zero, which I guess hasn't happened in a while. So, in the making of this, here's some of the generated art prompts. Area 51 active inference, Area 51 neural network, Area 51 active inference, and Area 51 active inference neural network. Just some interesting images coming out of stable diffusion. So as to some big questions that the paper is addressing and that one might be interested in to come to the paper. How can artificial neural networks be understood as generic optimization processes? And what is the correspondence between neural dynamics and modern statistical inference methods? Other big questions are about the history and next steps of the enmeshment of natural intelligence, e.g. neuroscience and artificial intelligence, as well as, of course, whether to even play into this kind of distinction at all and have different integrated intelligence frameworks. And one paper where any of the authors or anyone who has kind of resonated with this work is recent by Zador et al, 2022, Towards Next Generation Artificial Intelligence Catalyzing the Neuro-AI Revolution. And so this is a bunch of authors. And so it's interesting just to quote in terms of what some areas of discourse are saying right now, which is neuroscience has long been an important driver of progress in artificial intelligence, AI. We propose that to accelerate progress in AI, we must invest in fundamental research in neuro AI. So that's one way to lead some of the developments that are happening in the paper we'll discuss. What does it mean to be particular but generic? That's a phrase used in the paper. So maybe that's kind of a jumping off point. And then how can active inference help us understand the past, present and future? Here of the interface with neural networks, statistics and neuro AI. Here's the abstract. This work considers a class of canonical neural networks comprising rate coding models, where neural activity and plasticity minimize a common cost function. And plasticity is modulated with a certain delay. We show that such neural networks implicitly perform active inference and learning to minimize the risk associated with future outcomes. Mathematical analyses demonstrate that this biological optimization can be cast as maximization of model evidence, or equivalently minimization of variational free energy, under the…