Learning Generative State Space Models for Active Inference

hello everyone welcome to actin flab this is actin flab model stream number 3.1 it's may 27th 2021 and we're here with tim verbellen and ozone catal today we're going to have a model stream on some of their recent work on learning generative state-space models for active inference we're gonna have a presentation followed by some time for question and answer so please feel free to write any questions in the chat and we will get to them in the conversation so thanks again to tim and ozon for joining us today we're really looking forward to what you have to share so please take it away and thanks again so thank you daniel for having us um so i'm tim verde and together with uh rosencattel we will talk a bit on our paper on learning alternative state space models for active inference so we are both researchers at imek kent university in belgium if you want to know more about what we do you can read about it on our blog or follow us on twitter under under at the smart robots so before we dive in maybe a short introduction on on what we do what our goal is so basically we want to build intelligent robots so in our lab here in gans we have a really big pretty pretty large space where we have some room for robot manipulators on the one hand but also driving and flying robots on the other hand and we typically attach all kinds of sensors on these things and then we want to process the sensor information and infer some useful actions for these things and of course active inference is a cool methodology to try out and to further investigate and it's in this context that basically this this work is being done so why would you bother learning the state space so if you look at active inference papers from the recent years then typically you will find a figure like the ones that are on the slide so you have this figure of what is the kind of environments that that you're that you're modeling that you're investigating and then a whole description on how the model should look like so you define the state space and then possibly or in the case of discrete state spaces you you define the the so-called a b c and d matrix that's uh that defines the the likelihood model so what what is the likelihood to see a certain observation when you are in certain state or whether the transition models uh how do you transition from one state to another and so forth um and recently while we were doing this research research we saw some other people also thinking about yeah um can we do more learning using these these kind of novel deep learning techniques in these uh in these active inference methods so yet for example kai will suffer um who said uh you proposed to to basically learn kind of a state space for for the mountain car but then still he needed to explicitly encode some of the of the environment information in in the state vector there was also some work from from from baron millage who basically did some learning a bit more on learning the policy rather than the state space so um basically used um work of decision process with small state spaces where you could basically just deal with the the raw state space the the observations that the environment gave you where were basically suited as a state space and he could then he would then investigate how to learn the actions given given these states but so our work is basically on what if you don't know the state space what if your observations are high dimensional and you cannot really use this directly as your state space how do you define the model how you can come up with it and maybe as a concrete example suppose we have one of our driving robots in the lab so this this is a first-person view of the robots this is the kind of observation that you get is just a square of pixels and then yeah what's what is your state space in this case it might be an xy position on the map that might be something relevant for a robot as a state it might also be uh watch out you're approaching a cable getter so there…