Mike Litman
How to Evaluate an Agency That Builds
A TALK BY MIKE LITMAN

How to Evaluate an Agency That Builds

When the pitch becomes a prototype, the evaluation changes too.

Mike Litman · ~4 min read · 18 slides

$12.5bn

is what your agencies spend on pitching every year. Most of it is wasted.

FORRESTER 2023 YOUR BUDGET
THE CLIENT PROBLEM

You are evaluating performances, not capabilities.

The pitch shows you who is best at pitching. It does not show you who is best at making things. These are two different skills, and you are selecting for the wrong one.

THE A-TEAM PROBLEM
In the pitch
You see the senior team for 90 minutes. Polished slides. Rehearsed chemistry. The agency at its most performative.
On day one
The B-team arrives. Different people, different energy, different capability. The team that won the pitch is not the team that does the work.

A prototype sprint solves this. You evaluate the people who will actually do the work.

01
WHAT A PITCH ACTUALLY TELLS YOU

A pitch reveals who presents well. A prototype reveals who thinks well.

Presentation skill and problem-solving skill are different competencies. A deck can hide weak thinking behind strong design. A prototype cannot. Every decision is visible.

02
WHAT A PROTOTYPE REVEALS

The micro-decisions that matter.

The typeface they choose. The error message copy. The loading state. How they handle edge cases. What they leave out. These are taste decisions, and they only become visible when someone is building. No pitch deck has ever shown you a loading state.

03
THE DUE DILIGENCE CASE

A prototype sprint is better due diligence than any credentials deck.

You see how the team communicates under pressure. You see what they prioritise when time is short. You see whether they listen to feedback or defend their work. This is the chemistry test that actually predicts the relationship.

THE ECONOMICS
Running a Pitch Prototype Sprint
Timeline 4-6 months 2-4 weeks
Your team 8-12 people reviewing 2-3 stakeholders
Agency cost £50K-£200K unpaid £5K-£20K paid
You get Promises A working product
FASTER CHEAPER BETTER SIGNAL
THE PROOF POINT

Droga5 moved to paid pitches only. Their win rate tripled.

When both sides pay, both sides take it seriously. The agency sends its real team. The client engages properly. The output is closer to what the actual work will look like.

DROGA5 3X WIN RATE BOTH SIDES INVESTED
THE EVALUATION FRAMEWORK

Five things to evaluate in a prototype that you cannot evaluate in a pitch.

01
EVALUATE

Did they understand the problem or just the brief?

The brief tells you what the client asked for. The problem is what the client actually needs. A prototype reveals whether the agency understood the difference. Look at what they chose to build and, more importantly, what they chose not to build.

02
EVALUATE

How resourceful are they under constraints?

A two-week sprint with a small budget forces trade-offs. The choices they make under constraint tell you more about their judgement than any strategy document. Did they solve the problem elegantly or throw resources at it?

03
EVALUATE

Is there taste in the details?

Zoom into the edges. The transitions. The copy on the 404 page. The way the navigation feels on mobile. These are the decisions that reveal whether someone cares about craft or is just ticking boxes. Taste is not in the hero image. It is in the footnotes.

04
EVALUATE

How did they respond to your feedback?

During the sprint, you gave notes. Did they defend their position with evidence or fold immediately? Did they push back constructively or agree with everything? The best agencies will argue with you when they are right. That is what you want in a partner.

05
EVALUATE

Are these the people who will do the work?

In a pitch, the agency fields its best presenters. In a prototype sprint, the people building are the people you evaluate. There is no bait-and-switch. What you see is what you get. Ask yourself: do I want to work with these specific humans for the next twelve months?

HOW TO RUN A PROTOTYPE EVALUATION
1
Set the brief, not the format
Define the problem. Do not prescribe the solution. Let the prototype reveal how each agency interprets the challenge.
2
Pay for the sprint
£5K-£20K per agency. Two weeks. This is a fraction of what a traditional pitch costs you in internal time alone.
3
Stay involved
Check in on day 3 and day 7. Give real feedback. This is a collaboration test, not a presentation test.
4
Evaluate the work, not the walkthrough
Use the prototype yourself before the presentation. Form your own opinion. Then hear their rationale.
2-4 WEEKS PAID SPRINTS YOU ARE INVOLVED
THE SHIFT

Stop buying promises.
Start evaluating evidence.

The agencies worth hiring are the ones who can show you, not just tell you. The prototype makes that possible at a fraction of the cost and time of a traditional pitch.

Evaluate the work.
Not the performance.

hello@mikelitman.me

Based on original research · Sources: Forrester, EACA, IPA, ISBA, Bedford Group

Continue from slide ?