Principles of scalability and biological inspirations

[Laughter] so hello everybody oh can you just go on mute if you're not speaking sorry I'm just going to do a bit of an introduction um so welcome to the third um morph stream series the others are on YouTube if you've not had a chance to catch those so just a reminder of the purpose of these meetings are to educate and Inspire at the intersection of active inference and neuromorphic Computing um if you've not met me before I'm Sarah Hamburg so I'm predominantly a neuroscientist I did my PhD neuroscience at UCR which is where I first came across active inference um and I've just finished a postto in neurom morit Computing and I'm part of the Sab for the active inference Institute and I've think I've been involved with them since about 2020 when I was trying to work out earlier but I'm really pleased today that we're joined by Professor Annan sub money so he's an assistant professor in the Department of computer science at Royal Holloway London he's broadly interested in learning and intelligence so that's both algorithmic and biological his current research is focusing on understanding the principles of scalability in deep learning which is what he'll be talking about today sorry I'm just admitting some people um so he aims to use principles of scalability to build models that are more efficient and can scale up seamlessly his research draws inspiration from neuroscience and biology in his Quest To Build a Better and more General artificial intelligence and you can visit his website onand subber money for more information I'll pop that in the chat as well so his talk today is on principles of scalability and biological Inspirations he's going to talk about how current models scale and what we can learn from the efficiency of biological brains one of the central themes will be sparsity its significant role in scalable systems and its synergies with neuromorphic Hardware he'll present existing ideas based on spiking networks and recent work from his group which is focused on using various forms of sparity and distributed learning to improve the scalability and efficiency of our learning models so the format will be that the first part um arnand will speak and then we'll sort of have a more General open conversation between all of us um arand I don't know if you want people to in interrupt you with questions or not if you just want to take them at the end that's entirely up to you yeah I'm actually completely fine with taking questions during the talk cool okay right well without further Ado uh yeah we'd love to hear your presentation yeah thank you Sarah and thank you very much for the invitation uh so yeah like Sarah mentioned I'm going to be talking about principles of scalability and biological Inspirations uh so I've been working with computation neuroscience and machine learning for quite a long time and have been thinking about uh well I mean how intelligence Works to some extent but also really uh a big part of it is about understanding how various how biological models scale and by extension how artificial intelligence model scale and that's what I'm going to be talking about today so just to start off with uh things that we already know I mean so AI is really starting to become better than humans at lots of things right so this is kind of like a a plot of performance of various AI models in relation to humans and you see that handwriting recognition and speech recognition image recognition so this all of these have been like improving very very fast uh but recently things like language understanding and reading comparation have also in imp improved and become better than humans so there is really some kind of like a major movement going on in AI where things are really becoming uh better very fast and so the question is what is happening or why does it so work so well so there have been a lot of uh advances in the past do decades uh because we've started using deep neural networks with lots of layers and lots of…