In a randomised trial developers felt 20% faster with AI and were measurably 19% slower. Every prompt here produces something you can verify rather than something to trust.
The speed-up you feel is not always the speed-up you get
METR ran a randomised controlled trial with 16 experienced open-source developers across 246 real tasks in repositories they already knew well — on average about five years of familiarity with the codebase. Tasks were randomly assigned to allow or disallow AI assistance.
The developers forecast they would be 24% faster with AI. Afterwards, they estimated they had been 20% faster. Measured, they were 19% slower.
The result is narrow and should not be over-read: sixteen developers, mature open-source projects, tooling from early 2025, and expertise in the specific code being changed. It does not show that AI assistance is useless, and the same tools plainly help enormously on unfamiliar code, boilerplate, and one-off scripts.
What it does show is worth taking seriously anyway, because the perception gap is the finding. The developers were wrong about their own productivity by nearly forty points, in the favourable direction, on tasks they had just completed. Whatever is true for your team, your sense of whether it is working is not evidence.
That is the discipline this pack is built on. Every prompt here produces something you can check — a review against a stated dimension, a test that either passes or fails, a diff explained, a risk assessment with reasoning shown. None of them ask you to trust an output you cannot verify.
Understanding code you did not write
Explain Unfamiliar Code and Explain an Unfamiliar Codebase are the two prompts most likely to genuinely save time, because they operate exactly where the METR conditions do not apply — code you do not have five years of context on. Onboarding into a new repository, taking over an abandoned service, or reading a dependency's internals to work out why it does something strange.
Find the Root Cause of an Error From Its Stack Trace works from the trace to the likely cause, distinguishing the frame where the error surfaced from the place the problem was introduced.
Turn a Question Into a SQL Query and Write a Regular Expression From a Description cover two notations that are easy to read and hard to write. Both are verifiable against real input, which is the point — run it against the data before you trust it.
Review, tests, and the things reviewers miss
Code Review Focused on One Dimension is deliberately single-axis. Asking for a general review produces a general answer that catches style issues and misses the concurrency bug. Asking specifically about error handling, or about what happens when this input is null, or about the security properties of this endpoint, produces something a reviewer can act on. Run it several times with different dimensions rather than once with all of them.
Generate Unit Tests With Edge Cases puts the value where it belongs — in the cases you did not think of. Empty inputs, boundaries, unicode, concurrent access, the failure path of every dependency. A generated test that passes is worth nothing; a generated test that fails has just told you something.
Refactor Code Without Changing Its Behavior is scoped by its title and enforces the constraint that makes refactoring safe: behaviour preservation is a testable claim, and the prompt insists you have the coverage to make it before you start.
Assess the Risk of a Dependency Upgrade works through breaking changes, transitive impact, and what your code actually touches — because the risk of an upgrade has almost nothing to do with the size of the version bump.
The writing around the code
A meaningful share of engineering time goes on prose, and it is the part most reliably improved by a model that has the diff in front of it.
Write a Commit Message From a Diff and Write a Pull Request Description From a Diff both work from what actually changed rather than from what you intended to change — which is frequently the more useful description. The PR prompt covers the part reviewers most need and most rarely get: why, and what to look at first.
Generate Documentation and Docstrings From Code is written to document behaviour rather than restate the signature.
Generate Release Notes From Merged Pull Requests translates from the language of the change to the language of the user, which is the whole job and the reason most release notes are unreadable.
Translate Code From One Language to Another carries the strongest caveat in the pack. Translated code that looks right and carries subtly different semantics — integer division, string encoding, error propagation, mutability — is worse than no translation, and the prompt is written to flag those rather than paper over them.
Working with agents
Two prompts here are about the tooling itself. Write a Project Instruction File for a Coding Agent is the highest-leverage document most engineering teams have not written: the standing context a coding agent reads on every run — conventions, commands, structure, and the things that will otherwise be got wrong every single time. Cut the Cost and Latency of an AI Feature belongs to the AI and agent engineering pack, which covers the wider architecture, context and security questions.
Where this stops
Everything here produces output you must review. Generated code can be confidently wrong, generated tests can assert the wrong behaviour, and a security review by a model is a first pass rather than an assurance. Nothing in this pack replaces a human reviewer who is accountable for what ships, and nothing here should be trusted in a safety-critical, regulated, or security-sensitive path without a qualified person checking it. Be careful what you paste: source code, credentials, and customer data in a prompt are subject to whatever your organisation's policy and your provider's terms actually say.
Sources
- METR, Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity, July 2025 (arXiv:2507.09089) — 16 experienced developers, 246 tasks in familiar repositories; forecast 24% faster, self-estimated 20% faster, measured 19% slower