Watch, don't ask

You asked Marta how the Thursday update works. She told you, clearly and helpfully, and you wrote it down:

Export the report, check it against the system, update anything that has changed, mark it done.

Four steps. It sounds like a small feature. It is not what happens.

People cannot report their own behaviour accurately

This is not about honesty. It is a well-measured limit on what anyone can tell you about their own routine.

The clearest demonstration comes from research on digital media use, where you can compare what people say against what a device logged. A preregistered meta-analysis across 106 effect sizes found that self-reported use correlates only moderately with the logged reality, at r = 0.38, and concluded that self-reports “were rarely an accurate reflection of logged media use.” Not biased in one direction, which you could correct for. Just unreliable, in both directions.

What people give you when you ask is a reconstruction. They remember the shape of the task, the part they find annoying, and the version they would describe to a new colleague. What drops out is everything that has become automatic, which is precisely the part where your software lives.

And before this sounds like a claim about other people: when METR put sixteen experienced developers on 246 real tasks from their own repositories, they finished 19% slower with AI tooling while reporting afterwards that they had been about 20% faster. Experienced professionals, describing their own work, on the thing they do all day, with the sign wrong.

The move

Ask how it works, then watch it happen, and when the two accounts disagree believe the one you watched.

Note that this is not stop asking. The best systematic review of requirements elicitation found that “interviews, preferentially structured, appear to be one of the most effective elicitation techniques”, and that several fashionable alternatives, including card sorting and thinking aloud, tended to be less effective than simply interviewing people.

So keep the conversation. It is how you find out what matters, what they are worried about, and what they wish were different. Just stop treating the answer as a description of the process, because that is the one thing it reliably is not.

What to record while you watch

Sit slightly behind them, say nothing, and write down four things.

The order things actually happen in. Not the order they were described in. These differ more often than you would expect.

Every application they touch. Most workflows worth fixing leave your product at least once, and the leaving is usually invisible to everyone except the person doing it.

Where they hesitate. A three-second pause before clicking something is a decision they are making that your interface is not helping with. Those pauses are where the design work is.

What they do twice. Checking the same number in two places, re-reading a row, going back. Repeated actions are almost always compensation for something the system will not tell them.

WHAT SHE DESCRIBED               WHAT SHE DID

 1 export the report              1  open last week's spreadsheet first,
 2 check against the system          "to see what changed"
 3 update anything different      2  export this week's
 4 mark it done                   3  paste both into a third sheet
                                  4  sort by cost centre
 four steps                       5  eyeball the diff  <- 4 min, no tool
 one application                  6  alt-tab to the admin screen
 about twenty minutes             7  search each changed person by name
                                  8  edit, save, back  <- x40
                                  9  re-check three she wasn't sure about
                                 10  screenshot the sheet, paste into Slack
                                 11  mark it done

                                  eleven steps, three applications,
                                  fifty minutes, and step 5 is the job

Step five is the whole chapter. Nobody describes step five, because it does not feel like a step. It feels like thinking. It is also the only part that is genuinely hard, and it is completely invisible from the ticket, from the interview, and from your analytics.

What it costs

It is expensive and it does not scale. Twenty minutes of watching costs you an hour with the scheduling and the travel, and you can realistically do this a handful of times a quarter. Interviews are cheap and you can do ten. That asymmetry is real, and it is the actual reason most teams skip observation, rather than any argument about method.

Being watched changes what people do. Marta will narrate, apologise for her spreadsheet, and tidy up a step or two. You cannot eliminate this. You can reduce it by saying almost nothing, and by watching a second time, when the novelty has worn off.

And you will want to help. You will see step eight and know a keyboard shortcut that saves her ten minutes. Do not say it during the session. The moment you start improving her process, you stop seeing it, and the shortcut is worth much less than the eleven steps you came for.

Try this week

Ask one person to let you watch them do the thing you are about to build for. Twenty minutes, no preparation, no tidying up.

Say this, because the framing matters more than the request:

I don’t want you to explain it, I just want to watch you do it normally. Ignore me and narrate nothing. If you’d normally swear at the spreadsheet, swear at the spreadsheet.

Write down the eleven steps. Then compare them with the four you were told, and count what was missing.

Whatever occupies the largest single block of time in that list, and is not in the ticket, is what you should actually be building.