Choose the Right Model for a Task
Works out which model tier a task actually needs by defining the hardest case it must handle, then designs the head-to-head test that settles it on your own data. Use it before defaulting to the largest model, or before switching to a cheaper one to save money.
0 likes
0 dislikes
Sign in to rate this prompt
Prompt
You are an engineer who has run model bake-offs and knows that most model choices are made by reputation rather than measurement.
The task: {{task}}
Where it runs (product feature, batch job, internal tool, coding agent): {{context}}
Volume: {{volume}}
Latency requirement: {{latency_requirement}}
Budget constraint, if any: {{budget}}
What a bad output costs: {{cost_of_error}}
Candidates I am considering: {{candidate_models}}
Do this.
1. **Name the hard part.** What specifically makes this task difficult: long context, multi-step reasoning, precise instruction-following, tool use, structured output, domain knowledge, or writing quality? Models differ far more on some of these than others, and the answer determines whether tier matters at all. If the task is genuinely easy, say so — that is the finding that saves the most money.
2. **Set the requirements as thresholds, not preferences.** Minimum acceptable quality (and how you would recognize it), maximum acceptable latency at p95 not average, and cost per run at my volume. A requirement without a number is not a requirement.
3. **Design the bake-off.** 20 to 50 real inputs from my actual distribution, deliberately weighted toward the hard cases rather than the typical ones. Specify how each output gets scored, who or what does the scoring, and what counts as a meaningful difference rather than noise.
4. **Test the cheap one first.** Run the smallest plausible model before the largest, because if it passes, the comparison is over. Say which of my candidates that is and what would make it fail.
5. **Consider a split.** Would routing — cheap model by default, escalate on a detectable condition — beat any single choice? Name the condition that would trigger escalation and how you would detect it without a second model call.
6. **Plan for the model changing under me.** Models deprecate and get updated. What should I pin, what should I abstract, and what should my eval set catch when it happens?
Return: a recommendation, the test that would prove it, and the specific result that should change my mind.