[Deep] Learning Active Inference

All right, hello and welcome. It's August 5th, 2024. This is Active Inference Guest Stream 85.1, Deep Learning Active Inference with Love with David Blumen. So thank you, David, for the presentation and we'll look forward to the discussion. How we can use Active Inference to make an intelligent agent and some of the advantages and disadvantages and some issues with Active Inference and how I think we can overcome them. And you'll see why with Love in a little bit. But my background is I'm a software engineer and an AI researcher. I've worked for a long time at the intersection of scalable systems and machine learning. And I've been working on this project for about three years now. And I'm really excited to talk about it. So I think the holy grail of AI research right now is how do we make an AI agent that's both generally intelligent and doesn't destroy humanity. And pretty much I think most people are working on some facet of these two. And I'm really excited about the generally intelligent part and also not destroying humanity. So here's how I think we can do it. But first I want to talk about just like get our definitions. So what's an agent? And there's lots of people arguing about definitions. I'm going to use a really simple one, which is an agent is something that given a set of past observations produces the next action. So basically given what you're seeing now and what you've seen and experienced in the past, what should you do next? So a very simple formulation of an agent as a function. And by generally intelligent, I don't have a rigorous definition of general intelligence. I don't actually think there is one. But as an observer observing this agent, we should be able to say, hey, it's it's acting smart in a wide variety of environments that it hasn't seen before. And what that means is that it's able to figure out the environment dynamics, it's able to make mistakes and learn from them. It doesn't get stuck in the local optima, like knows when to explore, but then when it figures out regularities about the environment, it knows how to use them. It does this efficiently. It's not just like trying the same dumb thing over and over again. It makes plans, it can communicate and track with other complex intelligent entities in the environment. So it's a bunch of different behaviors. And I don't think there's a really clear way to quantify or like, talk about exactly what they look like, because different spaces in different environments are going to have a different idea of what an intelligent agent is doing there. So for now, just like think of it as this grab bag of capabilities. So where does active inference come in? Well, active inference in the environment is a way of thinking about an agent interacting with its environment as essentially a, it's a formulation where any agent can be seen as if it was acting in a self evidencing way. And what that means is that you can treat any object that maintains homeostasis over time, as if the inside of that object was minimizing free energy, meaning that it had some, some, you can think of it as having some generative model of its environment around it from an agent's perspective, its boundary, and it is trying to update its prior beliefs or its posterior beliefs over the boundary in order to minimize surprise. And so the way it does that is when it receives evidence from the environment, it adjusts its set of beliefs to concord with what it thinks it's gonna, what the environment is and what it's doing and where it is in that environment. But it's also actively navigating the environment in order to both get, gather more knowledge and improve its set of beliefs, as well as to move into parts of the environment that it thinks it's going to be occupying. And so this is a really beautiful way to capture what any agent is doing at any scale. So this formulation allows us to think about a biological cell or a robot or a person or an organization, a corporation, a country,…