From the course: Introduction to LLM Vulnerabilities
Join today to access over 24,000 courses taught by industry experts.
What are LLMs and how do they work
From the course: Introduction to LLM Vulnerabilities
What are LLMs and how do they work
- [Instructor] To understand LLMs, and how do they exactly work? We first need to start with tokenizers, and understanding what tokenizers is. In this case, we have here a piece of text that poses the question, what is a tokenizer? And what that exactly means is that a tokenizer is a process where we convert text like this phrase right here, into numbers. Large language models will work with text as input, and generate text as output. But how exactly does that happen? Well, it happens by processing a phrase like in this case, and segmenting that into smaller chunks. And in this case, for example, what would be one of those chunks, one of those pieces? And that will be converted into a numerical value. In that case, that what would perhaps be an numerical value like 318. All of those pieces would then essentially be converted to numbers and be put into a sort of an array. That is the way that a machine can then process those, and try to identify better what are some of these pieces…
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.