Choose Few-Shot Examples That Actually Help
Selects, orders, and stress-tests few-shot examples so they teach the boundaries of a task rather than one happy path — including when few-shot is the wrong tool and instructions would do better. Use it when output is inconsistent or drifts toward whatever the examples looked like.
0 likes
0 dislikes
Sign in to rate this prompt
Prompt
You are an engineer who has seen few-shot examples make a prompt worse as often as better, usually by teaching a pattern nobody meant to teach.
The task the model must do: {{task}}
The prompt as it stands: {{current_prompt}}
Examples I have or am considering: {{candidate_examples}}
The output shape I need: {{output_format}}
Where it currently goes wrong: {{failure_modes}}
Do the following.
1. **Check whether examples are the right fix.** Examples teach format, tone, and edge-case handling well. They teach reasoning and domain rules poorly, and they cost tokens on every call. If my problem is an underspecified instruction, say so and fix the instruction instead — that is the cheaper and more reliable answer, and it is frequently the correct one.
2. **Assess coverage of what I have.** Map my candidate examples against the real input space: typical cases, boundary cases, cases where the right answer is to refuse or return nothing, and cases that look like the task but are not. Name the gaps. A set of three examples that all show the same easy case teaches the model that the easy case is the whole task.
3. **Recommend a set**, with a specific count and the reasoning for that count. For each chosen example, say what it teaches that the others do not. Cut any example that is redundant — a shorter set of distinct examples beats a longer set of similar ones.
4. **Write the ones I am missing.** Draft the examples needed to cover the gaps, especially the negative and boundary cases. Make them realistic rather than tidy; examples that are cleaner than real inputs cause failures on real inputs.
5. **Order and format them.** Where they sit relative to the instructions, how each is delimited so the model cannot confuse an example with live input, and whether to show reasoning or only the final answer.
6. **Stress-test for accidental lessons.** Look at the set as a whole and tell me what unintended pattern a model might extract: a length, a hedging habit, a recurring phrase, an entity type, an implied ordering. This is where few-shot quietly goes wrong.
Return the final example block, ready to paste, plus a short note on what each example is carrying.