Textbook Group meeting for Parr, Pezzulo, Friston 2022 .

all right greetings everyone thank you for joining so we're in this first up of the third weeks for the cohort it'll be a bit of a grab bag so we'll see where it goes but a few people have written some first questions um and then we'll look at some prior questions see what else comes up all right so first I'd be interested in examples on how to train an active inference model what what would anyone say on this how is training an active inference model similar or different than potentially other kinds of models um hi uh I can say uh so um I was thinking like um there should be a world model which would be trained on the um on minimizing the variation of free energy uh as a loss objective so I mean I'm assuming that can be for for from from an uh like an RL perspective or something like from some collected trajectories that can be trained but but for the expected free energy uh I'm I'm not so confident like how H how how that can be deployed yeah thank you Frasier I would just I mean I've only successfully made maybe like two or three active inference models that are like non-toy examples but one thing that I found particularly helpful was or or Salient was that you had to really think about the different time scales of things that we're going to be training so you know I guess in in a machine learning context training refers to the updating of parameters of a model so you know and and as was just uh said by is it three I think it's three we need our our model to embody uh the environment in some way so that's going to be parameterized there G to be a bunch of parameters and we need to tweak those parameters in a machine learning setting um that happens at a much slower or at least a slower time scale than the actions that we need to decide to emit at each time point if it's a a discrete time model let's say so there's kind of two questions there's the question of what actions do we emit based upon expected free energy based upon maybe just vfe and then there's the question of okay what how do we also minimize VF with respect to our parameters so those two questions Loom large at least in my mind when I'm thinking about training an active agent cool yeah well there's there's a lot of pieces to to bring out here um one note is that variational basian methods like the variational auto encoder are also trained or updated using variational free energy minimization so the concept of using the evidence lower bound to to um bound and approximate surprise which is equivalent to maximizing model evidence that's not a new modeling trick from active inference part of the interesting thing though is bringing the perceptual and the action selection process into the unified objective and then using the evidence lower bound on that so so there's like some I mean it it will be cool to kind of pull out from the conversations and papers and get to the vend diagram so that's that's one thing is training a model based upon free energy minimization is something you'll find in a variety of machine learning methods um an interesting difference in terms of of at least like in terms of how chapter six differs uh from some other machine learning Pathways like nowhere in chapter 6 The the recipe for modeling does it say anything like go find a data set of positive examples and then tune your model so that it recapitulates the maximum performance on the positive data set so it's kind of interesting is like prior data can have a different role in the situation of training an active inference model so sometimes in machine learning you have a you have a basically a priori prior setting of a general or a large topology of the model like here's the architecture of the neural network and the layers or I'm training a 7 billion parameter Transformer architecture so you kind of have a large generic function approximator where the nodes are essenti not intended to have any semantic meaning and then during the training phase you pump cats and dogs with labeled…