A Neuro-Symbolic Approach to Language Understanding: Using an Ontology to Create A Language Model

Hello and welcome. This is ActInf Guest Stream number 32.1. It's January 10th, 2023. We're here with Adam Peace and we will be hearing a presentation followed by a discussion. So Adam, thank you very much for joining today. Looking forward to this and off to you. Okay. Thanks very much for having me. Looking forward to presenting a few bits of initial research as well as some background on long-term ongoing research into understanding language and trying to mimic some, just some elements of human thought. Everything I do is open source, so I've got some links for you there. I also have a pretty active YouTube channel, although it's been a little quiet lately. So there's a lot of material here that I won't be able to go into in depth. And so eager to answer questions when we get to the question period. And also would encourage folks to take a look at that channel where there's longer video format presentations about a whole host of topics that we'll probably touch on today. So I have a large ontology. I'll explain in a minute what I mean by an ontology that's called SUMO. And I'm also going to reference some work that Facebook has done called their baby corpus. So that's the background for the cartoons that I'm showing you. The main topic that I want to cover today is the notion of common sense auto-formalization. My objective is to be able to make statements in human natural language, just in English at the moment, ask questions also in natural language, and get answers. But get answers with explanation, something that modern question answering systems that we hear about each day in the popular press are generally not capable of doing. You might get an answer, but you don't know why. And knowing why and understanding an explanation is very important. So we're not talking about just information retrieval. We're not talking about Google, where you ask a question and you get a list of documents in which there might be an answer, or even Google's occasional capabilities for very specific sorts of questions, where they give you an answer, but they'll never tell you why. It's more like, you know, you ask what's the weather today, you get a nice concise link to the weather, but you won't get an explanation for why the weather is what it is. So what do I mean by ontology? Are you meaning to share, we see the first slide? You're still only seeing the first slide. Okay, I'm on slide three. So let's go back to, I'm not sure why that's no one showing up. But, uh, good. Just wanted to, looks good now. Common sense. Yeah, we'll stick with this format then instead of the slide presentation format. Thank you. Looks good. Okay. So here's what I just talked about. Um, so let me, uh, give you my definition for what is an ontology. Cause this is a word that at a certain point in recent history, maybe about a decade or two ago, got overloaded so much now that just about anything is called an ontology. And so for me to distinguish ontology from things we already had words for like semantic network or schema, um, I consider an ontology to be essentially a dictionary for computers to read. And that also requires a bit of unpacking. What is a dictionary for computers to read? Well, for me, it means minimally, uh, we've got to have terms, a set of terms that, that label things in our world. But most importantly, we have to have a set of definitions that are stated in a computable language. So just stating things in English or French, uh, that are definitions for words. Well, that's a dictionary. It's not an ontology. Um, and if we don't have definitions in a, some sort of mathematical language, a programming language of some sort, uh, then our machines are only going to have the barest abilities to manipulate or do things with those natural language definitions. So I think they have to be in a formal language to nail down meanings that we have shared meaning. So we have shared meaning between people that, that, uh, agree to use a particular ontology.…