Context windows and prompts

A model has no memory between calls. Each request starts from nothing. What it can use is the context window: all the tokens you send in one request. Google’s docs compare it to short-term memory.

In a chat app, the prompt the model really sees is usually several parts joined together:

• System instruction: your fixed rules (“You are a trip planner. Keep answers short.”).
• Conversation history: earlier messages, sent again every time.
• Extra context: documents, search results or tool results you add.
• The new message from the user.

Many Gemini models have context windows of 1 million tokens or more. Google’s docs give examples of what that holds: about 50,000 lines of code, 8 average-length English novels, or transcripts of over 200 podcast episodes. That’s a lot, but it isn’t free: you pay for every token on every call, and bigger prompts take longer.

Press Play to watch a short chat. Then switch Try it to Long chat: oldest turns dropped, and finally to keep a summary of key facts.

What to notice

1. The model only knows what’s in the window. No hidden memory, no other chats. If a fact isn’t in the prompt (and wasn’t in its training data), the model can’t use it.
2. “Memory” is your app re-sending history. The bot seems to remember because your code stores each turn and sends it again. See Training an LLM: chatting never changes the model itself.
3. What falls out is your choice. Dropping the oldest turns is simple but loses important facts. Summarising old turns, or storing key facts separately, keeps them in far fewer tokens.
4. Output counts too. The reply is generated into the same request. On Gemini 4 models, the 1M-token window is shared by input, thinking and output tokens. Earlier Gemini 3 and 2.5 models have separate input and output limits.
5. A big window isn’t perfect recall. Google’s docs report that models find a single fact in a long context very well, but accuracy drops when you need many separate facts from it at once.