Open Source AI: The truth is the first casualty in war

Hello and welcome everyone. This is Octave Inference Guest Stream 72.1. It's March 1st, 2024 and we're here with AutoMeta Alignment Lab AI. So welcome. Please feel free to introduce yourself and kick it off and we'll take it from there. Unmute and then go from there. Yeah, it would be really good if I did unmute first. Thanks. I appreciate the introduction. My name is Austin. I run Alignment Lab AI. We kind of build out in the open source and aim, at least I personally, we aim for kind of building solutions to problems before we have to deal with impacts. I think in general, it's probably pretty important that this AI thing goes about as well as we can get it to go for everyone. I think it's really important. There's quite a lot of outcomes, man, that we can really get from all this because I can say with a lot of confidence from the inside that the rabbit hole really just keeps going deeper when you start to look into the stuff. And it's meaningful because the rate that things are moving right now is not a temporary thing. Really, I don't think that we're moving as fast as we're going to be before it's all over said and done. And it's a good thing because so far, hugely positive impacts from my perspective. I think, yeah, I think that like overall, if it's going this well, this long, I mean, we're a year in, it's going to be, it looks better and better every day, I think, for the way that things are turning out. Okay. So many places to jump in. A year from what? Better according to what? Oh, fair. You know, a year from GPT, Prometheus bringing us the fires of Olympus and infinite free data to train whatever kind of model we want. I think that's really the big thing that has happened is this, the performative use of language by the models really allows for this nice kind of gluey middle ground to build data sets out for different use cases just by saying that you want them. Because prior, most things you needed to build classifiers by hand from scratch. And that alone is just one step of a larger pipeline. And I think that that is a barrier. It's like no longer there for like a lot of things, which is nice. Okay, let's take a step back and then re-approach. What were you working on as you turned to language models? Or how did you get into this space? So I've always had, I guess you could say like a bit of an obsessive nature. And I've always had a lot of involvement in my life with complex and challenging things. And at the time, I was just getting into like image models and Pythia and the earlier LLMs before Llama really leaked out. And that was when I first started to take the space like very seriously. I had dabbled a little bit and done a few little projects like text-to-speech models and things of that nature beforehand. But when the GPT-4 came out, and I was messing with the original version of that model when it first came out, that was like a very different experience. And I definitely stayed up for like a couple of days messing with it. When it first came out, I had like 100 messages every three hours you could talk to it for. And it was so much slower back then, that was plenty. And I don't know, I just thought it was really fascinating. And then I just kept doing it. And then I started making money doing it. And so I just kept doing it harder. And since then, I think all the original people that I first met up with in the open source, we still all talk to each other. And we just kind of fell in together, because that's just what happens, I think, when you're doing it for free. Okay, again, a lot of places to go, but maybe give a little bit of an overview for those who have or haven't worked on open source software or projects. Like, what does it really mean for one of these models to be open source? Are there different senses that people are using it? Yeah, so largely everyone's in the open source is, it's pretty, it's a weird split between machine learning engineers and people using it for like formal academic…