You don’t need to understand in detail how AI works in order to use it. But sometimes curiosity wins. And a deeper understanding often turns out to be very useful in practice.
Take the question of whether personal data is still personal data after it has been tokenized and vectorized. The answer: in principle, yes. Tokens are simply numbers in a fixed vocabulary, and the original text can be recovered from them one to one. The vectors aren’t a one-way street either: each token has a fixed vector in the model’s embedding table, which can be traced back to that token.
A deeper understanding also helps put into perspective what a language model actually does, which makes it a building block of good AI literacy. A language model doesn’t “think” in the human sense. It calculates, step by step, which word is most likely to come next.
One of our lawyers built an interactive visualization (with the help of AI) based on the open model Llama 3.1 8B. It shows what happens between input and output: from the sentence to tokens and vectors, through 32 layers, all the way to the next word. The presentation is kept (relatively) simple, with technical details and formulas for those who want to dig deeper.
Take a journey through the AI model. Click here.
No comments