Generalized decomposition of multivariate information

All right, hello and welcome. This is Active Inference Math Stream number 10.1, March 28th, 2024 or 29th, not exactly sure. And we're here with Thomas Farley discussing generalized decomposition of multivariate information. There will be a presentation followed by a discussion. So Thomas, thank you very much. Thank you for joining. Looking forward to this and everyone's questions. Yeah, thank you for having me. I'm really excited to get to talk about this with people who are also interested. And I think it's the 29th because I compiled this LaTeX last night was the last time I compiled it. So I think it's a one day out of sync. All right. And so yeah, so I'll be talking today about generalized decomposition of multivariate information. This is sort of walking through a paper that I recently published in PLOS One, and I'll have a link to that at the end of the talk if you want to read more or sort of interested in what I've covered here. And so we'll start with a little bit of a background, sort of the intuitions around information theory and then introducing these two ideas of the partial information decomposition and the partial entropy decomposition. And I can't see who's in the audience, but you know, if you are an expert in multivariate information theory, you can just go watch YouTube videos for five minutes or so while we get everybody else up to stream. And then once we've got sort of all the necessary mathematical machinery built, then I'll talk about how I can tell I've taken the PID and the PED and sort of generalize them to this thing I'm calling the GID, the generalized information decomposition, which is based on the Kohlbach-Liebler divergence. And once we've kind of built that, we'll talk a little bit about how the GID can be used to get insight into other information theoretic measures, you know, well-known things like the total correlation or the Tononi-Sporne-Edelmann complexity. And then finally, to show that it is, you know, truly a generalization of the PID, I'll actually walk through the derivation of the classic Williams and Beer bivariate partial information decomposition from the generalized information decomposition. And there are actually some kind of interesting sort of mathematical questions that kind of that you discover as you do that derivation that, you know, I'll talk a little bit about. And then I have a brief section at the end sort of talking about sort of future work, what we could do with this. And since this is the active inference stream, you know, I'll end with a little bit of discussion of sort of predictive coding and neuroscience. And then I would be really interested in talking in some way to people in the audience who might be more expert in active inference or free energy than I am, you know, because there, you know, maybe things that could be done with this that are not super obvious to me just because I'm sort of coming at this from a slightly different angle. So with that in mind, let's get started. Background. So, you know, this is the active inference stream. So I sort of assume that we're all familiar with sort of information theory and complex systems. You know, it's really, I think, become the case that information theory is sort of emerging as a lingua franca for complex systems. Right. It's nonparametric. It's model free, which makes it very useful for complex or, you know, nonlinear systems that are not well modeled just by, you know, linear regressions. It has some really deep and interesting connections to thermodynamics that I won't talk about here, but, you know, that's something I think is really interesting. And there's some very cool work pushing that forward. And then what makes it most relevant for complex systems is that it really elegantly handles multivariate interactions. Right. You know, it's hard to sort of imagine what a five way Pearson correlation might look like besides just a covariance matrix of pairwise correlations. But information theory can…