FORMINDEX: FORMIS INtegrated Database EXploration

All right. Hello and welcome everyone. This is ActiveInfraint stream number 5.1 October 16th, 2024. We're going to begin with a GitHub push. I'm going to check it's been updated in GitHub. This is also published on Zenodo. So it has a DOI. It's citable. And to show how to get a DOI on Zenodo, I'm going to make release 0.3 right now, adding a new tag 0.3. ActiveInfraint stream number 5.1. Publish it. Published on GitHub. Come back to Zenodo. And we can find it citable. It just takes a few minutes for it to archive it. All right. So here we are. I have some topics and sections to go through. And then if you're watching live, please feel free to write any comments and questions in the live chat. And this should be pretty interesting. There's going to be a lot of ant bibliographic methods, as well as some generative AI, synthetic intelligence methods, usage of perplexity, LLMs, and so on. Okay. So we began with a welcome. Pushed the live stream update to GitHub. We'll give a brief overview of the form index project. I'm using cursor version 0.42.3. Made the new version and the repo is listed here. So what is the form index project? Here we are in the form index the platform, which is linked in the video description. So the form index is a multi-stage project related to ant literature. So in ant literature, there's the formis or formi, depending on how you pronounce it. And this was an effort, is an effort, that curated citations related to ants. And it has completeness in taxonomic areas of ant studies. So a lot of rare ant taxonomy literature complete up to 1996 and then not as much after. And so in form index, I'm setting out on a dual journey to do an analysis of formis for its own internal sensemaking, to understand formis as a literature resource itself, and then also to look more broadly towards integration of knowledge methods for mirror mecology, for ant research, and then eventually even go beyond that with meta and format and integrating bioinformatics and so on. So in this first part of the stream, it's going to begin with the formis analysis, which is going to briefly move through the ant-specific parts about how the bibtext bibliography with 80,000 references is brought in, and then some of the visualizations that come out of the just looking at formis itself. Then the second part of the methods will focus on generative AI. And we'll get there when we get there. So first for the formis analysis. So first, Sanford sent 316 sample citations. And on October 8th, I read in those just to ensure I could get the format going. He then sent over the bibtext plaintext file with 80,000 records, which I brought in, in the script section. So first, I'll run this. So first, reading in formis, and visualize. We'll run them. So first, reading in formis, we'll run this. Read in formis. This was from another computer, so we'll see. Let's fix it on the go. Meanwhile, though, continuing on the report. This script reads into bibtext and stores the JSON format. The generate target bibliographies subsets the 80,000 row JSON database into bibliographies that mention that term. So this is what it looks like for bioenergetics. It's a JSON that's a strict subset of the overall. We'll focus more on running the code for the second part that I've more recently done on this computer. The visualize formis.py. Let's see if this one works. Also, this one doesn't work, but for the all formis database and for every target bibliography, some kinds of images are made. So here's the distribution of records. Here's histogram of record types. So there's 60,000 articles, 11,000 tech reports, 5,000 collections, 3,500 PhD theses, 1,592 books, one miscellaneous. Here's authors by number of citations. anon. Coming through. Coming through. J.C. DeLavis only slightly behind. Publications per year. So as mentioned earlier, this is not the overall number of publications per year in Google Scholar or literature database. This is just for formis. And this reflects what could…