What is an LLM?
A large language model does one thing. It looks at the text so far and predicts the next token, then does it again. Watch it happen, and change the temperature yourself.
ChatGPT, Gemini and Claude can seem to understand you, reason, and write code. Underneath, each one is a large language model (LLM) doing the same small job over and over: given some text, predict what comes next.
It doesn’t plan the whole answer first. It predicts one small piece of text (a token), adds it to the text, and predicts again. A paragraph is just hundreds of these predictions in a row.
Press Play below to watch one sentence get finished. Then move the temperature slider while the probabilities are on screen, and press Resample to see a different continuation.
- Tokenize
- Token IDs
- Embeddings
- Transformer
- Probabilities
- Sample
- Append
The weather in Chennai is
These appear once the transformer has run.
The weather in Chennai is
This is the prompt. Press Play to see how a large language model works out what comes next.
Token IDs, embeddings, scores and attention weights here are illustrative. The softmax and sampling are the real maths.
The five ideas to take away
- Tokens, not words. Models read and write tokens: whole words, pieces of words, or punctuation. Pricing and context limits are counted in tokens too.
- Everything becomes numbers. Each token becomes an ID, and each ID becomes an embedding (a long list of numbers). The model only ever does maths on those numbers.
- Attention is how it uses context. Inside the transformer, each token weighs how much every other token matters. That’s how “weather” and “Chennai” shape the word that comes next.
- The output is a probability for every token. The model doesn’t output an answer. It outputs how likely each possible next token is, and a sampler picks one.
- Temperature controls risk. Low temperature keeps picking the safest token, which is predictable and repetitive. High temperature takes chances, which can be creative or nonsense. For agents and factual answers, keep it low.
Why this matters for agents
When an ADK agent “decides” to call a tool, it’s this same process. The model predicts the tokens of a function call instead of the tokens of a sentence. Everything in the later tracks (tools, memory, evaluation) is built on top of next-token prediction.