Write an Eval Set for a Prompt or Agent

Design a small, concrete eval set to test a prompt or agent before you change it — cases, inputs, and checkable pass criteria.

0 likes 0 dislikes
Sign in to rate this prompt

Prompt

    Help me build an eval set to test a prompt or agent, so I can catch regressions before shipping changes.

What the prompt or agent is supposed to do:
{{purpose}}

The prompt, or a description of its inputs and outputs (optional):
{{prompt}}

Produce:
1. **10–15 test cases** covering the main use cases, important edge cases, and known failure modes. For each: a short name, the input, and what a good output must (and must not) do — as a checkable pass criterion, not a vibe.
2. **A few adversarial cases** — ambiguous, out-of-scope, or tricky inputs that probe where it's likely to fail.
3. **How to score** — for each case, whether it can be checked automatically (exact / substring / format) or needs a judge (human or LLM), and the rule to apply.

Keep criteria concrete and objective enough that two people would grade a response the same way. If the intended behavior is underspecified, list the assumptions you made so I can correct them.

Like this prompt?

Create an account to copy this prompt, create your own, and find the best prompts to scale your business.