Prototype in the medium you will ship in

The advice you have heard is that prototyping in your real stack gets you better feedback than prototyping in a design tool. I am going to make a version of that argument, but first I have to tell you that the evidence for the usual version is bad, and that if you use it in a meeting somebody may know.

Medium does not change the feedback you get

Three controlled comparisons have tested this and all three found no meaningful medium effect.

Walker, Takayama and Landay tested paper against computer prototypes with 28 participants, and their conclusion is unambiguous: “Designers can choose the most practical prototyping medium because user-testing feedback is equally good with either.”

Sefelin and colleagues compared paper against computer-based low-fidelity prototypes with 24 subjects, and found users preferred the computer version. Virzi, Sokolov and Karis ran two experiments back in 1996 and found low and high fidelity surfaced the same usability problems.

So the fidelity of what you put in front of a user is largely not the variable. Somebody testing a paper sketch will find what somebody testing a working build finds.

What the medium does change is which questions you can ask

The useful framework here is Lim, Stolterman and Tenenberg’s, and it separates two things that get conflated. Material is the medium. Resolution is the fidelity. They are independent dials, and you choose each one deliberately.

Their governing rule is what makes this chapter’s argument work:

“The best prototype is one that, in the simplest and the most efficient way, makes the possibilities and limitations of a design idea visible and measurable.”

Measurable is the word to hold onto. A prototype is a filter: it makes some qualities of an idea visible and hides the rest.

And here is the thing about being a developer. The questions you have are mostly not the questions a picture can answer.

Does this list stay usable at four thousand rows, which is what our biggest customer has? What happens between the click and the response on a real connection? What does this look like when the third field comes back null, which it does, about twice a week? Can we actually get this data in one query, or is this design quietly a three-second page?

None of those is a taste question, so none of them is answerable in a design tool. They are answered by running the thing with real data on a real connection, which means the medium that answers them is your shipping stack.

The move

Choose the medium by which question is open. If your open question is about feel, sketch it. If it is about data, latency or failure, build it thin and real.

OPEN QUESTION                        MEDIUM THAT ANSWERS IT

is the flow understandable?          paper, a sketch, a Figma frame
which of three layouts reads         any of them. the evidence says
  better?                              feedback is equally good
is the wording right?                a screenshot in Slack

does it hold up at 4,000 rows?       the real stack. only the real stack
how slow is it on a real             the real stack
  connection?
what does the error state            the real stack
  actually look like?
can we even get this data?           the real stack, in about an hour

  most developer prototypes are built to answer the top
  three questions, which were never the ones in doubt.

What it costs

Building in your stack is slower to change than a drawing, and this is the real objection. Moving a button in Figma takes a second. Moving it in a component with tests attached takes longer, and if the question genuinely is where should the button go, you have chosen an expensive medium for a cheap question.

And a working prototype gets mistaken for a working product. This is the failure mode that costs teams weeks. It runs, it uses real data, and somebody asks whether it can go out on Friday. Move thirteen’s discipline applies here: write down what is deliberately fake, and say it out loud every time you show it.

Also, do not read the disconfirming studies as permission to skip prototyping. They say the medium does not change the feedback. They do not say the prototype does not matter.

Try this week

Before you build anything for the next piece of work, write down the one question you are actually unsure about. One sentence.

Then pick the medium that answers that. If you cannot say what the question is, that is the finding, and it usually means you are prototyping to look busy rather than to learn something.

If the question involves real data, give yourself two hours and a hardcoded route in your own codebase. Not a design, not a mockup: a page that runs one real query against a copy of production data and renders whatever comes back, ugly.

Most of the time that page answers the question in an afternoon and changes the design before anybody has drawn it.