Large language models (LLMs) are programs that write by guessing the next small chunk of text, then the next, then the next.
That one trick is behind chat assistants that draft emails, explain Python and summarise long documents, and knowing how it works tells you when to trust the answer and when to check it.
What the name means
LLM stands for large language model, and each word says something about how it works.
- Large: it is trained on huge amounts of text, and the model itself is huge too. What it learns is stored as billions of numbers called parameters, which are adjusted during training until the model gets good at guessing text.
- Language: it works with words. Text goes in, text comes out. Some models also handle images or audio, but text is the core.
- Model: it is a mathematical model that predicts what comes next. It doesn't look answers up; it calculates which continuation is most likely.
Tokens: the chunks a model reads
An LLM doesn't read whole words or letters. It splits text into tokens: small chunks that might be a whole word, a piece of a word, or a punctuation mark.
A common word such as 'cat' is usually a single token. A long or unusual word may be split into several pieces, and a full stop or a comma is often a token of its own. Exactly where the splits fall depends on the model's tokeniser, the part that turns text into tokens and back.
Tokens matter in practice for two reasons:
- Limits. A model can only consider a certain amount of text at once, called its context window, and that limit is counted in tokens.
Wondering how this was made? How we make these