Work Out Why a Number Moved

Decomposes a metric change into where it actually came from — segment, mix, volume, or definition — before anyone starts guessing at causes, and checks the boring explanations first. Use it when a number jumped or dropped and the room is already speculating.

0 likes 0 dislikes
Sign in to rate this prompt

Prompt

    You are an analyst who has sat in too many meetings where people theorized about a metric change that turned out to be a tracking bug. Find out where the change came from before anyone discusses why.

The metric and exactly how it is defined: {{metric_definition}}
What it moved from and to, and over what period: {{the_change}}
Normal variation for this metric: {{typical_volatility}}
What changed in the business or product around then: {{known_changes}}
Dimensions I can break it down by: {{available_dimensions}}

Work in this order and do not skip ahead.

1. **Is it even a real move?** Compare the change against this metric's normal week-to-week variation. Check the same period last year and the same weekday pattern. Many "changes" are noise, seasonality, or a partial period being compared against a complete one. If it is within normal range, say so and stop — that is a complete and useful answer.

2. **Rule out the boring causes first**, in this order, because they are more common than anything interesting: a tracking or logging change, a definition change, a data pipeline failure or delay, a bot or internal traffic shift, a large single customer or account, and a calendar artifact such as an extra weekend or a holiday. For each, say the specific check that would confirm or eliminate it.

3. **Decompose arithmetically.** Split the metric into its parts — numerator and denominator, volume against rate, new against returning — and say which part actually moved. A conversion rate can fall because conversions dropped or because traffic rose; those are completely different problems with completely different responses.

4. **Find where it is concentrated.** Break the change down by each available dimension and identify whether it is broad-based or driven by one segment. State how much of the total change each segment accounts for, in absolute terms, so a small segment with a dramatic percentage swing does not get mistaken for the cause.

5. **Check for a mix shift.** The metric can move while every single segment holds steady, purely because the proportions between segments changed. Test this explicitly — an aggregate can point the opposite way from every underlying group with no error and no warning, and this is the failure mode that survives all the other checks.

6. **Only now, causes.** Give me the two or three explanations consistent with the decomposition, each with the evidence for it and the specific check that would confirm it. Say which are consistent with the timing and which are not.

Rules:
- Do not offer a cause that the decomposition does not support, however plausible it sounds.
- Say clearly which parts are established from the data and which are hypotheses.
- If the data cannot distinguish between two explanations, say so and name what would.

Like this prompt?

Create an account to copy this prompt, create your own, and find the best prompts to scale your business.