Chain-of-Thought Prompting Explained (With Examples)

Ask a language model to multiply two three-digit numbers and blurt out the answer, and it will often trip. Ask the same model to work through the multiplication one place value at a time, and the accuracy jumps. That gap is the whole idea behind chain-of-thought prompting: giving the model room to reason before it commits to a final answer. When a problem has moving parts that depend on each other, the reasoning is not decoration. It is the thing that gets you the right result.

This article walks through the main variants of the technique, shows where each one earns its keep, and points out the situations where reasoning steps just waste tokens. If you are still building your foundation, the fundamentals guide is a good companion read.

Why Stepwise Reasoning Changes The Output

A model generates text one token at a time, and every token it produces becomes part of the context for the next one. When you force a direct answer, you compress a multi-step problem into a single leap. The model has no scratch space, so it pattern-matches toward something that looks plausible and frequently lands near the answer rather than on it.

Reasoning steps change the shape of that generation. Each intermediate conclusion the model writes down becomes a fact it can lean on for the next step. A hard problem turns into a chain of easier ones, and the model only has to be locally correct at each link instead of globally correct in one shot.

This helps most on tasks with these features:

• Arithmetic and word problems where an error in one operation cascades
• Logic puzzles with constraints that interact
• Multi-hop questions that require chaining several facts together
• Planning tasks where order and dependencies matter

Zero-Shot: The One-Line Trigger

The lightest version costs you a single sentence. You append an instruction that nudges the model into a reasoning mode before it answers. The exact phrasing matters less than the intent, so treat it as a lever you can rephrase for your domain.

The model will typically establish the price per group, count how many groups are in 21 pens, compute the total cost, then subtract from 20. Compare that to a bare question, where it might grab the numbers 21 and 20 and produce something confidently wrong.

You can steer the reasoning too. Instead of a generic trigger, tell the model *how* to think: "First list the constraints, then eliminate options that violate them, then pick what remains." Domain-specific scaffolding usually beats a one-size instruction.

Few-Shot: Teaching By Worked Example

Zero-shot leans on the model's own habits. Few-shot shows the model exactly what good reasoning looks like by including one or more solved examples, complete with their intermediate steps. The model imitates the pattern of the demonstrations, not just their answers.

Because the worked example spells out its steps, the model continues in the same voice: it will name the reading days, count how many fall in 21 days, and multiply. The demonstration does double duty, showing both the answer format and the reasoning granularity you expect. If you want a deeper comparison of the two styles and when each wins, see few-shot vs zero-shot prompting.

Keep your examples close to the real task. A model shown geometry reasoning will not automatically transfer that discipline to a scheduling problem, so match the domain of your demonstrations to the domain of your query.

Decomposition: Break The Task Apart

Sometimes the smartest move is to stop treating the problem as one prompt. Decomposition splits a complex request into named subtasks, each of which you can solve and verify on its own. You either ask the model to list the subproblems first, or you drive them yourself across separate calls.

A research summary, for instance, decomposes cleanly:

• Extract the key claims from the source
• Check each claim against the supporting text
• Group related claims into themes
• Write the summary from the verified themes

Solving each stage in isolation gives you inspection points. If the summary is wrong, you can see whether the extraction or the grouping failed, rather than staring at one opaque block of output. This structured approach is a core theme in advanced prompt engineering, where chaining and decomposition often carry more weight than clever single prompts.

Self-Consistency And The Final Check

A single reasoning chain can go off the rails and still sound confident. Two techniques guard against that.

Self-consistency samples several independent chains for the same question, each with a bit of randomness, then takes the majority answer. If eight of ten reasoning paths land on 42, you trust 42 far more than any lone attempt. It costs more calls, so reserve it for questions where a wrong answer is expensive.

Self-check asks the model to audit its own work before finalizing. A closing instruction does a lot here:

That last instruction turns the model's own output into something it reviews rather than something it merely produces. It catches a surprising number of slips, especially arithmetic and off-by-one errors in time or counting problems. A dedicated builder like the chain-of-thought prompt tool can generate this reasoning-plus-verification scaffold for you, and the prompt optimizer helps tighten the wording once you have a draft.

When To Skip It

Reasoning steps have a cost. They spend tokens, add latency, and on the wrong task they can actually hurt by inventing justification for a fact the model already knew cleanly. Skip explicit chain-of-thought when:

• The task is a direct lookup or single fact ("What is the capital of Peru?")
• You need a short classification label with no ambiguity
• The output is creative and has no correct answer to verify
• Latency matters more than the last few points of accuracy

Forcing "let's think step by step" onto a simple retrieval just pads the response and occasionally talks the model out of a correct instant answer.

Reasoning Models Shift The Balance

The newest reasoning models handle much of this internally. They generate a hidden deliberation pass before responding, so bolting a generic "think step by step" onto them adds little. The reasoning is already happening where you cannot see it.

What still pays off is structure and verification. Telling the model *which* dimensions to consider, in what order, and asking it to run a final self-check gives you leverage even when the internal reasoning is strong:

• Specify the criteria the answer must satisfy
• Ask for the answer in a checkable format (a number, a table, a labeled decision)
• Request a short verification pass against your stated criteria

You are no longer teaching the model to reason. You are aiming its reasoning at your problem and giving yourself a way to confirm the result. That framing keeps chain-of-thought useful long after the raw "think step by step" trick stops moving the needle.

References

• Chain-of-Thought Prompting (arXiv)
• Prompt Engineering Guide