How AI Works · 10 minutes

Tokens, Context & Memory

Why can AI handle long documents but occasionally lose track of something said earlier? These three ideas explain a lot.

Tokens

Text is processed in pieces called tokens. One token may be a word, part of a word, or punctuation. Tokens are not the same as words.

Context window

The amount of information the model can consider at one time. Very long conversations or files may exceed this limit.

Memory

Some AI products can retain or retrieve useful information across chats. This is different from the text currently in the conversation.

A simple analogy

Tokens: Pieces of the material. Context: What's on the desk right now. Memory: Notes saved or retrieved for later. The exact behavior depends on the product and settings.

Live demo: A conversation with limits

  1. Ask AI to summarize three short paragraphs you provide.
  2. Ask it to use one detail from the paragraphs in a new email.
  3. Discuss what could happen if the conversation becomes very long or the earlier text is no longer in context.
Why tokens matter: They affect model limits, document capacity, and (in many developer or API services) billing. Most everyday users do not need to count tokens manually.

Quick exercise

Ask: “Explain tokens and context windows as if you're teaching a 12-year-old, using a backpack analogy. Keep it to four sentences.” Then ask one follow-up question.

Do not teach a fixed tokens-per-word rule. Tokenization varies by language and content. Do not promise that the product remembers everything.