ELBOing Stein: Variational Bayes with Stein Mixture Inference

Hello, welcome. It is April 10th, 2025. We're in Active Inference Math Stream 12.1 with Ola Running, going to be discussing Elbowing Stein, Variational Bayes with Stein Mixer Inference. So, thank you for joining. Looking forward to this presentation and discussion. Great, thanks for having me. I'm Ola Running presenting the work I've done with Eric Nolestek, Christoph Lai, Frederick Smith and Thomas Hammerruck on Variational Bayes with Stein Mixtures. So let's get into it. So, this is the plan. We'll talk about patient inference and particle-based methods and particular Stein-based methods which came about in 2016. I'll just go over an overview of what these are to get everybody up and then the key problem with them which is the underestimated variance as our models get high dimensional. Then I'll talk shortly about our approach which is Stein Mixture Inference. The new thing here is introducing a neighborhood to each other particles. We'll talk a little bit about that and then evidence that this mixture actually helps with the issue of dimensionality, of course, of dimensionality. And finally, Stein Mixture Inference is a black box inference engine in the probabilistic programming language of Pyro. So I'll plug that a little bit at the end. Yeah, so just to set the stage here. So making predictions and sort of modeling systems is ubiquitous. So the group I'm in works with structural biology, but I'm looking to applications in robotics and potentially finance as well. And sort of at an abstract level, what we're looking for is a trustworthy systems. We want something that's accurate when it meets, when our data meets our expectation and uncertain when it's either surprising, ambiguous or missing. So illustrating it here by where we have a data generating process dotted line, and then we have a method which is trying to infer to infer the process. This is the red line. And so we only give it sufficient data in the green regions. And what we're looking for is something that captures the process in the green region and gets uncertain in this in between region. And so on the left hand side, you would have something that's accurate, but it's but it's overly confident because it the uncertainties is collapsed in the in between region. And the right hand side is what we're what we're what we want. So here, the blue region could be any potential curve. And it's like 95 or 90% HDI. So it's like the likelihood there would be in here. Good. Yeah. And so just to make it slightly more concrete. So in prediction, you might have something like a overconfident, overconfident mobile, mobile robot. So if you for example, have a robot navigating North a tree, it will have a sort of uncertainty about its orientation and position. But as you put it into new terrain, for example, by removing a tree, you would want this uncertainty to spread out so that you can invoke emergency protocols and then overconfident protocol would sorry, overconfident robot might lead to crashes because it's there's a disambiguity between its actual location and where it believes it's at. And this is a this is a we have a talking with some underwater robotics labs and they say that this is an issue in one out of 250 missions. And this can be quite expensive with robot costing between 20,000 euros and 10 million depending on the size of the robot or deep sea robots. Another issue. So that was sort of when we're talking about prediction, but when we're talking about modeling protein structures, three dimensional protein structures have regions which are conserved. So they stay ordered so they don't fluctuate very much. But it's it's a it's a molecule interacting with the solvent around it. So the whole thing will move slightly. And then you have some regions which are unstructured and highly disordered, and they will be moving around. So if you want a description of the structure, it's not enough of the three dimensional structure to protein, it's not enough to know just…