Causal Inference via Predictive Coding

Hello and welcome. It's ActiveInference Guest Stream number 51.1 on July 28th, 2023. We are here with Tomaso Salvatore and we will be having a presentation and a discussion on the recent work, Causal Inference via Predictive Coding. So thanks so much for joining. For those who are watching live, feel free to write questions in the live chat and off to you. Thank you. Thank you very much, Daniel, for inviting me. I've always been a big fan of the channel and I've been watching a lot of videos, so I'm quite excited to be here and be the one speaking this time. So I'm going to talk about this recent preprint that I put out, which has been the work of the last couple of months. And it's a collaboration with Luca Vincetti, Amin Makarak, Beren Millic and Thomas Lukasiewicz. And it's basically a joint work between Versus, which is the company I work for, the University of Oxford and TUVN. So during this talk, I will, this is basically the outline of the talk. I will start talking about what predictive coding is and give an introduction of what it is, a brief historical introduction, why I think it's important to study predictive coding, even, for example, from the machine learning perspective. I will then provide a small intro to what causal inference is. And once we have all those informations together, I will then discuss why I wrote this paper, what was basically the research question that inspired me and the other collaborators, and present the main results, which are how to perform inference, so intervention and counterfactual inference, and how to learn the causal structures from a given data set using predictive coding. And then I will, of course, conclude with some, with a small summary and some discussion on why I believe this work can be in fact impactful and some future directions. So what is predictive coding? Predictive coding is in general famous for being a neuroscience inspired learning method. So a theory of how information processing in the brain works. And very formally speaking, the theory of predictive coding can be described as basically having a hierarchical structure of neurons in the brain. And you have two different families of neurons in the brain. The first family is the one in charge of sending prediction information. So neurons in a specific level of the hierarchy send information and predict the activity of the level below. And the second family of neurons is that of error neurons. And the error neurons, they send prediction error information up the hierarchy. So one level predicts the activity of the level below. This activity has some, this prediction has some mismatch, which with what actually going on in the level below. And the information about the prediction error gets sent up the hierarchy. So, however, predictive coding is, was actually not burned as a neuroscience, as a theory from, from the neurosciences, but it was actually initially developed as a method for signal processing and compression back in the fifties. So the, the work of Oliver, Elias, which are actually contemporary of, uh, Claude Shannon, uh, of Shannon. They realized that once we have a predictor, a model that works kind of, that is well in predicting data, sending messages about the error in those predictions is actually much cheaper than sending, uh, the entire message every time. So, and this is how predictive coding was born. So as a, uh, as a signal processing and compression mechanism in information theory back in the fifties, it was actually in the eighties, uh, that, that he became that exactly the same model was used in, uh, in neuroscience. So with the work from Mumford or other works that, for example, explain how the, how the retina process information. So we get prediction signals from the outside world, and we need to, to compress this representation and, uh, and have this internal representation in our neurons. And the method is, uh, very similar, if not equivalent to the one that was used, the, that was…