Textbook Group meeting for Parr, Pezzulo, Friston 2022 .

all right hello everyone it is July 11th and we're in the second discussion on chapter two so before we go into any of the questions does anyone just want to bring up anything or just start with anything about chapter two um Ali and then anyone else who raised their hands uh thank you yeah I just uploaded uh the equation to 0.5 walkthrough uh in the um I guess I put it in the equations table if I'm not mistaken but equation 2.6 needs a little bit more fine-tuning before it's ready but uh I'm sure it'll be ready for the next week so I think I put it at the top of the notes section of equation 2.5 so if anyone wants to uh but yeah that's it foreign looks awesome can we maybe uh go over it or can you just walk through it or yeah sure yeah uh well first of all um we began with um just basic definitions uh such as some fundamental definitions such as the Shannon entropy and then to derive uh the first line we put we substitute the Shannon entropy term uh into the uh I mean the first line of equations so uh and we can see there's uh an interesting parallel between uh this kind of formulation between energy and entropy and also the path integral formulation and variation of preology but the essential difference here is that here the first term represents only the energy but for the path integral it can be best described as the energy constraint uh so that's um more of a side note there and then for the second line of equation 2.5 we get but before going into um the second line just let me unpack the first line a little bit more so for for the people who who are not familiar with a callback Library Divergence uh we put the definition of callback library at the top of the um or yeah at the top of the page one and then uh by using uh James inequality we get uh the Callback the lighter Divergence bounded by the expectation of the log p over q and that allows us to substitute the Shannon entropy term into the expectation of the log p over Q so that was the essential move for unpacking the first line of equations so I mean without Jensen's inequality we wouldn't exactly get the the exact term of the first line of the equation because obviously the order of the log operator and the expectation operator is different than uh we need here so yeah that's basically a straightforward algebraic manipulation um and the the second line of equation 2.5 or you know the terms the trade-off between complexity and accuracy similarly can result from uh substituting uh I mean by expanding uh the first line of the equation and then substituting some of the term as the Callback Library Divergence and then we get the complexity term uh and the rest will be the accuracy so this what it means is that as pointed at the bottom of the page the greater complexity is uh in other words the more one needs to change beliefs to explain observations the Lesser predictive accuracy will become so in this case variational free energy uh would be just the minimization of the complexity or at the same time maximizing the accuracy and again it can be compared to the path integral formulation it's almost identical to the path integral formulation of the trade-off between complexity and the accuracy uh all right so for this for the third line of equation 2.5 uh we try to somehow get the trade-off between the Divergence and the evidence but in this case evidence is just uh something that acts as um as the bound or the uh evidence lower bound uh so it's something that's always greater than or equal to surprise right so by unpacking the um the last line of equation 2.5 into these two distinct terms or the this trade-off between Divergence and evidence uh we are allowed to have this um elbow or evidence lower bound or in other words the negative upper bound that would enable us to regard BFE as the optimization parameter uh or over the parameter to be optimized rather than uh trying to achieve the exact value of uh free energy but just some optimization parameter which would be…