The brief has changed. Agencies are showing up with URLs instead of decks. With working products instead of mood boards. With prototypes that you can actually use, not presentations that describe what they might one day build.
This is good news for clients. But it creates a problem. Most procurement processes, most evaluation scorecards, most pitch frameworks were designed for a different kind of agency. The evaluation criteria made sense when everyone showed up with slides and a reel. They need updating.
Here is what to look for when an agency pitches with something real.
The A-team problem
Every agency has an A-team. They are the people who pitch. Senior, articulate, impressive in a room. They know the right things to say about brand strategy and they know how to make you feel like you are in safe hands for 90 minutes.
Then the project starts. The A-team moves on to the next pitch. The people who actually do the work show up on day one, and they are not the people you met.
This is not a secret. Clients know it happens. Agencies know clients know it happens. Nobody talks about it directly because the pitch process is designed around performance, not production, and performance requires the best performers.
When an agency builds something as part of the pitch process, you are seeing who will actually build your thing. That is worth more than any credentials reel.
Five things a prototype reveals
Not all prototypes are equal. Here is what to interrogate in the work itself, rather than in the walkthrough of the work.
Problem understanding. Did they get the actual problem, or did they just solve the brief as written? The brief describes symptoms. The best agencies diagnose the underlying issue and build to that. A prototype that solves a slightly different problem than the one you specified is usually a good sign, not a bad one. It means they thought.
Resourcefulness under constraints. A prototype built in two weeks with limited information reveals how a team operates when things are not fully defined. That is exactly what a real project feels like. Watch how they handled the gaps. Did they make reasonable assumptions and flag them clearly? Did they ask sharp questions or broad ones? Did they invent where they needed to, or wait to be told?
Taste in the details. Strategy is invisible in a pitch deck. It is unavoidable in a prototype. Look at the typeface choices. Read the error message copy. Notice what happens when something goes wrong, or when a user takes an unexpected path. Notice the loading states. These micro-decisions are where genuine understanding of your brand, your audience, and your problem space becomes visible. You cannot write your way to good taste. It either shows up in the work or it does not.
How they respond to feedback. A prototype gives you something to push against. Ask them to change something, preferably something that goes slightly against their recommendation. Watch the response. Do they capitulate immediately? Do they argue the point with evidence? Do they find a third option you had not considered? The answer tells you everything about what the working relationship will actually feel like.
Whether these are the people who will do the work. Ask directly. Who built this? Who would be on the account? Match names to faces. If the prototype was built by a senior team but the day-to-day would be handled by juniors, that is a conversation worth having before you sign anything. The prototype sprint removes ambiguity here in a way that no credentials session ever could.
How to run it
The evaluation process does not need to be complicated. The goal is to get signal faster, at lower cost, with higher fidelity than a traditional pitch.
Start with a brief and chemistry session. One day. Meet the people. Align on the problem. No credentials reel, no agency history. Just: here is what we are trying to solve, here is what success looks like, here are the constraints. Ask them how they would approach it. Listen to what questions they ask back.
Then run a paid sprint. Two weeks is usually enough. The fee sits somewhere between £5K and £20K depending on the scope and the agency's seniority level. This is not a token payment to make the pitch feel fair. It is the beginning of a real working relationship. You get a working version of the idea. They get skin in the game. Both sides are invested in quality, not just in winning.
Stay involved during the sprint. Not to micromanage – to evaluate. How do they communicate? How do they handle ambiguity? Do they bring you in when they hit a decision point, or do they disappear and resurface with a finished thing? The process is part of the product.
At the end, evaluate the work, not the walkthrough. Ask stakeholders to use it independently before the presentation. Gather unmediated reactions. The presentation will be good – agencies are excellent at presenting. The product will tell a more honest story.
Six weeks, not six months. Brief, sprint, decision. Done.
The economics
Procurement teams worry that paid pitches are expensive. They are not, relative to the alternative.
A traditional multi-agency pitch with four agencies, full creative response, tissue sessions, and formal presentations absorbs enormous internal resource. The client team's time alone – briefing, attending, evaluating, managing the process – often exceeds the cost of a paid sprint several times over. That is before accounting for the false signal a polished deck generates, or the cost of a misaligned appointment discovered three months into a relationship.
A paid prototype sprint gives you better information, faster, at lower total cost. Two weeks of real work versus six weeks of process theatre. A working product versus a polished PDF. The actual team versus the pitch team.
What this changes on your side
This model only works if the client side adjusts too. A few things worth noting for those running the evaluation.
Write a better brief. The prototype sprint amplifies brief quality. A vague brief produces a prototype that solves the wrong problem. A sharp brief – specific problem, specific user, specific success criteria – produces a prototype that is genuinely useful as an evaluation tool and potentially as a starting point for the real work.
Give them access. Not unlimited access, but enough to make good decisions. One conversation with a real customer. One session with the team who lives with the problem daily. Agencies building in a vacuum build for the brief, not for the reality. The brief is always incomplete.
Resist the urge to over-evaluate the aesthetics. A two-week sprint is not a finished product. Judge the thinking, the taste, the problem framing, the responsiveness. Not the pixel perfection of something built under time pressure.
The right question
The purpose of a pitch has always been to answer one question: are these the people I want building things with me?
A prototype answers that question directly. A deck answers a different question: are these people good at making decks?
Those two questions have always had different answers. The industry is finally catching up to that fact.
The full 18-slide deck on this is at mikelitman.me/evaluate. And if you want to understand the agency side of the argument, the companion piece is at mikelitman.me/prototype.
