Free Energy Projective Simulation (FEPS): Active inference with interpretability

Hello and welcome. It's December 17th, 2024. We're here with Josephine Pizam and colleagues discussing free energy projective simulation. There will be a presentation followed by a discussion. So thank you to all the authors for joining. Looking forward to it. Take it away. Thank you. Hi, and thanks for having us. Today I wanted to present the paper that we have recently made public on the archive. You can have the number here, which treats the free energy projective simulation active in France with interpretability. And our goal there was to combine an existing framework for agency called projective simulation with the free energy principle. And active inference where we have an agent that is able to learn from his experience in the world, gathering percepts and remembering a history of actions to learn a representation of its environment and behave adaptively in its environment by performing active inference to determine its behaviors. I want to, I have structured my presentation in three main points. First, I want to introduce projective simulation to the community. Then I want to introduce the formulation and the aspects of active inference analysis that I have used in these works, especially the notation will be important. And then the main body of the presentation will deal with the free energy projective simulation, how, what is it architecture, how does it work, how do we train it, and then some numerical analysis about it before I conclude. So first, what is projective simulation? The goal of projective simulation is to model intelligent agents with a framework that is rooted in embodied cognitive science. So now what I want to do is to look at the keywords of this sentence and understand what we mean behind those words, starting with the notion of intelligence for an agent, where what we really mean here is that the agents we consider are able to perceive the environment with, to perceive their environment and are able to influence and change their environment by acting on it. And in doing so, they are settling matters in the environment in a flexible way. And then the agent, a broad definition for it, or its support would be a system that has a memory. As I said, I want my agent to be able to remember its experience in the environment. And it's also able to interact with this environment. And that leads us then to embody cognitive science, where instead of having the agent learning from a set of predefined rules that we as designers would give to the agent, such that it learns high order concepts, the agent has to learn from the information it has gathered from its interaction history with the environment. That is how it has perceived the environment and the action it has taken on this environment. And now I have spoken about memory earlier, and I want to make a precision for this intelligence. It should also, the agent should also be able to adapt to situations that it hasn't experienced yet by comparing it to its existing moments in memory and to be able to compare them and see which elements of its past history are useful to adapt to unprecedented situations. Now I want to dive into the way we are modeling the agent to implement projective simulation. The agent is engaged in a precept action loop with the environment in that it has two types of subsystems. A sensor system is able to perceive precepts and stimuli from the environment. The agent is able to act on the environment and change its state through its actuators. And the key of projective simulation, the place where projective simulation will be important, is the memory of the agent that is decoupled from its physical interface with the environment. And that should allow the agent to reflect such that it can display some more complex behaviors in the environment going beyond stimuli response behaviors. The idea here is that the agent is able to simulate possible scenarios using its memories in order to plan or deliberate for its next action. So this…