Section 2

How models actually work

Prediction, training, and limits.

4 lessons25-question quiz

2.1

Predicting the next token

6 min read

When a model writes, it does it one token at a time. At each step it looks at everything so far and produces a probability for every possible next token. Then it picks one, adds it to the text, and repeats. Sentence by sentence, that loop is the whole show.

blue62%

clear14%

falling6%

green1%

After "The sky is", a model might rank next tokens like this

Because it is choosing from a ranked list rather than always taking the top one, the same prompt can produce different answers. A setting called temperature controls this: low temperature makes the model play it safe and pick high-probability tokens (more consistent), higher temperature lets it take chances (more varied and creative).

Key idea

Text is generated token by token from a probability list. That is why answers vary, and why "temperature" changes how adventurous the output is.

Key terms

Temperature: A setting that controls how random the model's choices are.
More
Low temperature favors the most likely tokens (consistent, predictable). High temperature allows less likely tokens (more varied, sometimes more creative or more error-prone).
Probability: A score for how likely each possible next token is, given the text so far.

2.2

Where knowledge comes from

6 min read

A model's apparent knowledge comes entirely from patterns in the training data it saw. Nothing is looked up while it answers. This has two big consequences that trip up newcomers.

It has a knowledge cutoff

Training happened at some point in the past, so the model's knowledge stops at a cutoff date. Ask about something that happened after that date and it simply does not know, though it may still answer confidently, which is a problem covered later in this section.

It is not online by default

A plain model is not browsing the web as it replies. If a system can fetch live pages, check the weather, or read your files, that is because a tool was connected to it (covered in the Building with AI section). Without tools, everything comes from frozen training patterns.

Key idea

Knowledge = frozen patterns from training, up to a cutoff date. No live lookup unless a tool provides it.

This is also why two models can disagree on recent facts: they were trained on different data at different times.

Key terms

Training data: The text (and other data) a model learned patterns from.
Knowledge cutoff: The point in time after which the model has no built-in knowledge.
More
Anything that happened after the cutoff is unknown to the base model unless a tool fetches it at answer time.

2.3

The context window

6 min read

The context window is the amount of text a model can hold in view at one time, measured in tokens. It includes your prompt, any files or history you provide, and the answer being written. Think of it as the model's short-term working memory for this conversation.

Every model has a maximum window size. When a conversation grows past it, the earliest text falls out of scope. The model does not "forget" in a human way; that text is simply no longer in view, so it cannot influence the next token.

In the window	Not in the window
Your current prompt	Facts learned only during training
Files or text you pasted in	Earlier messages that scrolled past the limit
The recent conversation	Anything on the internet (without a tool)

What lives inside the context window

Key idea

The context window is working memory measured in tokens. Past its limit, earlier text drops out and stops affecting the answer.

This is why a long chat can start "forgetting" what you said at the top, and why pasting a huge document can push other things out of view.

Key terms

Context window: The maximum amount of text (in tokens) a model can consider at once.
More
It holds the prompt, provided files, conversation history, and the response being generated. Exceed it and the oldest content is dropped from view.

2.4

When models get it wrong

6 min read

Because a model generates plausible-sounding text rather than looking up facts, it can produce statements that are fluent, confident, and wrong. This is called a hallucination. It is not lying, which implies intent; the model is doing exactly what it was built to do, which is continue the text plausibly, and a plausible continuation is not always a true one.

Where hallucinations show up

Specific facts, dates, and numbers, especially recent ones past the knowledge cutoff.
Citations and quotes, which can be invented but look real.
Niche or obscure topics where the training data was thin.

Confidence is not a signal of correctness. A model can be equally fluent whether it is right or wrong, so its tone tells you nothing about accuracy.

Key idea

A hallucination is confident, fluent, and wrong. Treat model output as a strong draft to verify, not as a settled fact, whenever the answer matters.

The Using AI well section turns this into a habit: how to check what a model tells you before you rely on it.

Key terms

Hallucination: When a model states something false as if it were true.
More
It happens because the model produces plausible text, not verified facts. The fix is to check important claims against a trusted source.

Section 2 quiz

25 questions. Pass at 75% to master this section. Retakes are unlimited, and the quiz is where the learning sticks.

Section 2 quiz · How models actually workQuestion 1 of 25

How does a model produce a sentence?

Back to AI Foundations