Deep temporal Models of the Translation Process

Hello and welcome. It is October 30th, 2023. We're here in Active Inference Guest Stream number 62.1 with Michael Carle and friends. Today the talk is called Deep Temporal Models of the Translation Process, a Translation Agent Grounded in Empirical Data. We're going to have a presentation by Michael followed by a panel discussion. So if you're watching live, please feel free to write any questions in the live chat. Otherwise, we're looking forward to this presentation and discussion. So thank you again for joining Michael and friends to you for the presentation. Yeah, good morning. So I talk Deep Temporal Models of the Translation Process. I would like to outline an idea for an architecture for the translation process. And to do this, I have three points here. First, I would like to introduce translation process research, what this is and the data that we have collected. I will show the data acquisition tools, visualization and segmentation. Then I will come into an architecture that is based on these partial observable Markov decision processes and that uses the free energy principle. And in the end, I will talk about an experimental implementation and initialization of those parameters. Okay, before starting talking about translation process research, here is a small contextual contextualization of this field of research. So it started around 40 years ago. And it's all about investigating the thing what happens in the mind of the translator. So how do translators produce translations? How do they have, how is it possible to have two languages in one mind? What makes translations of those languages? What makes translations of those languages? What makes translations of those languages? So this type of research started around 40 years ago with CRINGS around the beginning of the 80s. And they were using introspection and think aloud protocols. That was a laboursome manual analysis that went along with this. Then 10 years later, aunt luke jacobsen devised a key logging tool called translog that would record keystrokes translator would sit in front of an editor and type in the translations and the keystrokes would be logged then 10 years later in 2005 eye tracking devices were added to this device so with an eye tracker the gaze on the screen would be recorded in time and with a keylogger the output so we have an eye tracker recording where on the screen the translator looks and the key logging would log where what the translator writes and so we have somehow an input and an output and then in 2010 around that time i started collecting all these data into a database putting the data that we had collected by then in one con one database and a couple of years ago we started having this with a browser interface and python and r interface and so i like to call this translation data analytics because it resembles a little bit data analytics so we have these these python tools and the large database that i will talk later a little bit about and so this our research is on behavioral measures means the keystrokes and gaze data but there's of course also other kinds of data acquisition like brain activities eeg fmri and so on but i will talk about this part where we have key key logging data and eye tracking okay so how does it work so this is a screen this is a picture of the kind of setup that we have used quite frequently in this research so you see here on that screen translog 2 that is an editor so in the on the top of this window here you have a source text window that translator is supposed to translate translate translate in the lower editing box here and by typing there the keystrokes would be logged in time so every keystroke it comes with a time stamp and one could then look into the rhythm and when in the rhythm of translations when does the translator type fluently and so on when do they pause and then in the addition we have here an eye tracker we have light bulbs here this light is reflected in the eyes of a translator and…