Chain-of-Thought and Advanced Reasoning Prompting Techniques

Ask a model to think step by step and it often gets math, logic, and multi-hop questions right that it otherwise botches. Here's how that trick works, what's built on top of it, and why it needs a lighter touch in 2026 than it did two years ago.

What chain-of-thought prompting actually does

Chain-of-thought (CoT) prompting means asking a model to produce intermediate reasoning steps before its final answer, instead of jumping straight there. The technique comes from a 2022 paper, Chain-of-Thought Prompting Elicits Reasoning in Large Language Models, which showed large models solving arithmetic, commonsense, and symbolic reasoning problems did dramatically better when prompted to show their work.

Why it helps is mechanical, not magical. A model generates one token at a time, conditioned only on context so far. Ask for a final answer directly and it must compress all the reasoning into whatever precedes the answer token — often nothing. Writing out intermediate steps gives it more computation to work with, and each step becomes context for the next. For single-hop factual questions this buys nothing; for multi-step word problems, logic, or planning, it's often the difference between right and wrong.

This is foundational to the toolkit in the complete guide to prompt engineering, and interacts with prompt structure — see anatomy of a great prompt for where reasoning instructions fit among the other building blocks.

Zero-shot vs. few-shot CoT

Zero-shot CoT is the simplest version: append a phrase like Let's think step by step to your prompt, with no worked examples. This was formalized as its own finding — the instruction alone, without examples, recovers much of CoT's benefit on a standard (non-reasoning) model.

A store had 137 items, sold 42, and received 60 more.
How many does it have now?

Let's think step by step.

Few-shot CoT goes further: you show a worked example (question, short reasoning trace, answer) before the real question. This pins down not just that the model should reason but how — step granularity, notation, where to double-check itself. It generally beats zero-shot on harder or idiosyncratic tasks, at the cost of prompt length. Reach for zero-shot first since it's nearly free, and upgrade to few-shot when the task needs a specific reasoning pattern mimicked, like a particular way of breaking down a legal clause.

Self-consistency: sample, then vote

Self-consistency (Wang et al., 2022) layers a decoding strategy on top of CoT: instead of one path via greedy decoding, you sample several independent paths at nonzero temperature, let each reach its own answer, then take the majority. The intuition: a correct answer is reachable by multiple lines of reasoning, while wrong answers are more idiosyncratic to whatever mistake was made along one path — so agreement across samples is a decent proxy for correctness.

Practically: run the same CoT prompt N times (commonly 5–20), extract the answer from each, pick the mode. The tradeoff is blunt — roughly N times the tokens and latency of a single call — and it only helps when the task has a well-defined, extractable answer to vote on, not open-ended generation with no majority to compute.

Decomposition: least-to-most and problem breakdown

Where CoT reasons through a problem linearly in one pass, decomposition approaches split it into sub-problems first. Least-to-most prompting (Zhou et al., Least-to-Most Prompting Enables Complex Reasoning in Large Language Models) works in two stages: prompt the model to decompose the problem into a sequence of simpler subproblems, then solve them in order, feeding each solution into the context for the next. This matters for easy-to-hard problems where later steps genuinely depend on earlier results and one monolithic chain tends to lose track or skip steps.

More generally, problem decomposition just means instructing the model to enumerate subtasks before tackling them — useful for multi-part analysis or breaking a coding task into files and functions, either in a single prompt (list the subtasks, then solve each) or as separate orchestrated calls with focused context per subtask.

Tree-of-thought and ReAct, briefly

Two more advanced patterns are worth knowing by name, with clear eyes about their cost. Tree-of-thought (ToT) (Yao et al., 2023) generalizes CoT from a linear chain into a branching search: at each step the model generates multiple candidate next-steps, an evaluator scores or prunes them, and the search continues down the most promising branches, with optional backtracking. It's built for problems where a single reasoning path gets stuck — puzzles, planning, real dead ends — and needs meaningfully more orchestration and tokens than linear CoT, so it fits hard, high-value problems, not routine ones.

ReAct (Yao et al., 2022) interleaves reasoning with actions — the model reasons, calls a tool, observes the result, and reasons again. It's the pattern underneath most tool-using agents: reasoning lets the model plan and recover from surprises, and actions let it get real information instead of hallucinating it. It's less a competitor to CoT than its extension into agentic settings, with the cost profile of any agent loop attached — more calls, more latency, more places to go wrong.

Neither is a default reach. Know them by name, and judge whether the machinery earns its keep for the task at hand.

How native reasoning models change the calculus

Everything above was developed against standard, non-reasoning models — the kind that generate a response token by token with no separate thinking phase, so the only way to get intermediate reasoning was to prompt for it explicitly in the visible output.

Models with built-in extended thinking or reasoning modes — Claude's extended thinking, OpenAI's o-series and GPT-5 reasoning modes, and similar systems — work differently. They generate their own internal reasoning tokens before the final response, effectively doing CoT (and often something closer to search over multiple approaches) as part of how they were trained, not as a prompting trick you supply. Anthropic's own guidance is direct: don't hand-hold these models with think step by step or elaborate reasoning scaffolding — give the task and constraints and let the model decide how to reason through it.

Bolting the old techniques onto a reasoning model isn't just redundant, it can actively hurt: over-specifying reasoning steps can anchor the model to a worse strategy than one it would have chosen itself; verbose CoT instructions eat into visible or budgeted output without adding new reasoning capacity, since the model was reasoning internally regardless; and few-shot reasoning examples written for non-reasoning models can conflict stylistically with a reasoning model's own process, adding noise rather than guidance.

The practical shift: describe what you want and what constraints matter, and trust the internal reasoning phase to handle the how. That's the core argument in how to prompt reasoning models without getting in their way — the failure mode of over-prompting is easy to fall into out of habit.

When CoT helps, when it wastes tokens, and common pitfalls

It earns its keep when: you're on a standard (non-reasoning) model and the task has multiple dependent steps — math, multi-hop lookups, structured analysis; you need visibility into how the model reached an answer, for debugging; or the task benefits from explicit decomposition because subproblems build on each other.

It's wasted effort when: you're on a native reasoning model, where narrating a script just gets in the way; the task is a single lookup or classification with no multi-step structure, where think step by step just adds latency; or you want self-consistency-style voting on a task with no clean, extractable answer.

Common pitfalls: treating CoT as a universal quality boost — it targets reasoning depth, not factual accuracy, and a model can produce a fluent, wrong chain of reasoning as easily as a fluent, wrong answer. Trusting a model's stated reasoning as a faithful trace of what happened internally, rather than a generated artifact. Stacking techniques indiscriminately without checking whether the task justifies the multiplied cost. And forgetting that few-shot examples teach format along with reasoning, so sloppy examples produce sloppy reasoning regardless of model quality.

Chain-of-thought and its descendants remain some of the highest-leverage tools in prompt engineering, but highest-leverage no longer means always on. Picking the right one — or choosing none of them and trusting a reasoning model's own process — is itself part of the skill. For where this fits alongside the rest of the discipline, see the complete guide to prompt engineering in 2026.