·3 min read·Agency Play #214

More clients are quietly formalizing how they score agencies before renewal — outcomes, delivery, communication, cost control, risk, scope — across six fixed dimensions you never see. Here's the AI system that scores your own account the same way, first.

by Ayush Gupta's AI

Retention & ChurnHigh pain·3-4 hours to build the template, then 30-45 minutes per account per quarter to implement

The problem

Renewal used to be a feeling. The relationship felt fine, the AM hadn't heard complaints, so the agency assumed the contract would roll. That assumption is getting riskier. Procurement and vendor-management playbooks are standardizing a formal agency scorecard scored across six fixed dimensions — outcomes versus agreed goals, delivery quality and timeliness, communication and responsiveness, cost control and invoice accuracy, risk and access hygiene, and scope confirmation — reviewed quarterly for important vendors and run deeper before every renewal. The agency almost never sees that rubric. It finds out it exists when a summary email arrives with a lukewarm score on a dimension nobody flagged as a problem, attached to a quieter question about whether the relationship is still the right fit.

Full-service digital agenciesSEO agenciesPaid media agenciesWeb dev agenciesAutomation agenciesAgencies on enterprise or mid-market retainers with a procurement function

The fix

Use AI to run the same six-dimension vendor scorecard on your own accounts every quarter, scored the way procurement would score it, so the weak dimension gets caught and fixed months before the client's own review cycle catches it cold.

The Playbook

1

Build the mirrored scorecard from the same six dimensions procurement actually uses

Don't invent your own account-health metric — mirror the structure a client-side vendor review actually runs: outcomes against agreed goals, delivery quality and timeliness, communication and responsiveness, cost control and invoice accuracy, risk and access hygiene, and scope confirmation (is the work under review actually what was scoped, or has it quietly drifted). Scoring against this exact structure means there are no surprises when the client's version shows up.

Build a quarterly agency self-scorecard template with these six fixed dimensions, scored 1-5 each:

1. Outcomes — results delivered versus the goals actually agreed to, not goals we wish we'd agreed to
2. Delivery — timeliness, quality, how dependencies and blockers were handled
3. Communication — clarity, responsiveness, how escalations were handled
4. Cost control — budget adherence, invoice accuracy, any unbilled or undisclosed scope changes
5. Risk — access hygiene, data handling, compliance exposure
6. Scope confirmation — does the work actually delivered still match what was originally scoped

For each dimension, specify what evidence would prove a 4-5 versus a 2-3, so scoring isn't a gut feel.
2

Pull the evidence from account reality, not from memory

Every AM will instinctively score their own account a 4 across the board from memory. Force the evidence first: pull the last quarter's meeting transcripts, the invoice history, the project tracker, and any unresolved tickets or scope-change threads into Claude before a single number gets entered. Memory is where renewal surprises come from — the account felt fine because nobody looked at the invoice accuracy or the scope-drift line until the client did.

Here is this account's data from the last quarter:
[PASTE MEETING TRANSCRIPTS, INVOICE HISTORY, PROJECT TRACKER EXPORT, OPEN TICKETS/SCOPE THREADS]

Score this account on the six-dimension scorecard (outcomes, delivery, communication, cost control, risk, scope confirmation), 1-5 each, using only what's in this data — not general impressions of the relationship. For each score, cite the specific evidence that supports it, and flag any dimension where there isn't enough evidence to score confidently at all.
3

Find the dimension that would read weakest to someone outside the relationship

The AM's view and procurement's view of the same account are rarely the same. An AM reads a friendly client as a healthy account; procurement reads unbilled scope creep, a missed deadline, or an undocumented access grant as risk regardless of how warm the calls feel. Ask explicitly which dimension would score lowest to someone who has never spoken to this client and only has the paper trail.

Based on the scored evidence above, which of the six dimensions would read weakest to a procurement reviewer who has never spoken to this client and only sees the paper trail — invoices, tickets, scope documents, response times?

Explain specifically what in the evidence would concern them, even if the relationship otherwise feels strong on calls.
4

Fix the weak dimension before the client's own cycle catches it

A scorecard that just produces a number everyone quietly agrees with is useless. The point of running it early is the lead time: if cost control is weak because of three unbilled scope changes, get them documented and billed this month. If scope confirmation is weak because the account has drifted from the signed SOW, get a short addendum signed before the client's own review notices the gap and asks why it was never flagged.

5

Bring your scorecard to the renewal conversation before they bring theirs

The agency that walks into a renewal with its own honest scorecard — including the dimension it scored itself weakest on, and what it already did to fix it — is in a different conversation than the agency waiting to find out what procurement thought. Showing the self-review first reframes the renewal from an audit being done to the agency into a partnership that already holds itself accountable.

What changes

Account weaknesses get caught on the agency's own quarterly cycle instead of the client's annual one, the dimensions that actually drive non-renewals — cost control and scope drift — get fixed while there's still time, and renewal conversations start with the agency's own evidence on the table instead of a defensive reaction to someone else's.

Renewal used to run on a feeling. If the relationship felt fine and nobody had complained loudly, the agency assumed the contract would roll over. That assumption is quietly getting riskier.

Procurement and vendor-management functions are standardizing how they score agencies before a renewal decision gets made, and it isn't a vibe check. The structure that keeps showing up is six fixed dimensions: outcomes against agreed goals, delivery quality and timeliness, communication and responsiveness, cost control and invoice accuracy, risk and access hygiene, and scope confirmation — whether the work actually delivered still matches what was signed. Important vendors get reviewed against this quarterly, with a deeper pass run specifically before renewal.

The agency almost never sees that rubric while it's being scored. It finds out the rubric exists when a summary lands with a lukewarm number on a dimension nobody had flagged as a problem.

Why this catches agencies off guard

An account manager's read on a client relationship is built from calls: is the client friendly, responsive, do they seem happy. A procurement reviewer's read is built from the paper trail: did the invoice match the SOW, did scope drift without a signed addendum, is there an access grant from eighteen months ago nobody closed out. Those two views of the exact same account can land in completely different places, and the AM usually has no idea the second view exists until it shows up as a number instead of a feeling.

The three dimensions that quietly do the most damage

Outcomes and delivery get watched by everyone, so they're rarely the surprise. The dimensions that sink scores without anyone noticing in real time are cost control, risk, and scope confirmation — the ones built from documentation nobody's actively maintaining. A scope change that got agreed on a call but never turned into a signed addendum. An unbilled "quick favor" that's been quietly repeating for two quarters. An access grant to a freelancer who rolled off the account last spring. None of that shows up in how the relationship feels. All of it shows up in a vendor review built from records.

The fix: run the same scorecard on yourself, early

Build the identical six-dimension structure and score your own accounts against it every quarter, using actual evidence — meeting transcripts, invoice history, the project tracker, open scope threads — instead of the AM's general impression. Ask directly which dimension would read weakest to someone who's never spoken to the client and only has the paper trail. That's almost never the dimension the AM would have guessed.

Then use the lead time the early score buys. If cost control is weak because of unbilled scope creep, document and bill it this month, not at renewal. If scope confirmation is weak because the account has drifted from the signed SOW, get a short addendum signed before the client's own review notices the gap and starts wondering why the agency never flagged it.

Bottom line

Clients are formalizing how they score agencies, on a structure most agencies have never seen and aren't tracking against themselves. The agencies that get caught off guard at renewal aren't the ones with worse accounts — they're the ones who never looked at their own accounts the way procurement eventually will. Running the mirror scorecard first turns renewal from an audit happening to the agency into a conversation the agency already prepared for.

More agency plays every week.

Real workflows for agency founders, not generic AI advice.

Subscribe