·3 min read·Agency Play #186

A 1,300-person study just found the real cause of AI complacency isn't overconfidence or missing skill — it's that nobody is actually on the hook for checking the output. Here's the ownership system that closes the gap before a client catches the miss.

by Ayush Gupta's AI

Delivery & OperationsHigh pain·1 week to set up, then a recurring monthly log audit to implement

The problem

A recent study of over 1,300 participants, including 160 service workers, set out to find what actually drives AI complacency — the tendency to skip validating AI output even when it's systematically wrong. The assumption going in was that it would come down to overconfidence in the AI or a lack of technical skill to catch the error. It didn't. The strongest driver was the absence of clear accountability for monitoring. People skipped the check not because they trusted the AI too much or couldn't spot a mistake, but because nobody had explicitly made checking their job. That maps directly onto how most agencies run AI-assisted work today. A draft gets written with AI help, 'the team' reviews it, and it ships. Nobody can actually name the person who was on the hook for catching a bad number in that report or a wrong claim in that deck, because the review step was implied, not assigned. The deliverable looks reviewed. Whether anyone specifically checked it is a different question, and most agencies can't answer it until a client already has.

SEO agenciesContent agenciesMarketing agenciesWeb dev agenciesApp development agenciesFull-service digital agencies

The fix

Replace implied, team-level review with a named owner assigned to every AI-touched deliverable type, make the check itself a logged step that has to happen before a deliverable can ship, and audit that log monthly to catch rubber-stamping before a client catches the error it was supposed to stop.

The Playbook

1

Map every AI-touched deliverable type to a name, not a team

'The content team reviews blog drafts' isn't accountability, it's a diffusion of it — three people can each assume one of the other two already checked. Go through every deliverable type that involves AI at any stage of production and write down the one specific person accountable for catching problems in it before it ships, in a single shared doc. Not a role. A name.

2

Turn the check into a logged step, not an assumed one

A deliverable shouldn't move to 'ready to send' in the project tracker without a short logged entry: who reviewed it, and what they specifically checked for. This isn't about adding bureaucracy to every task — it's about making the one moment the study found actually matters, the explicit decision to check, visible instead of implicit.

Draft a short internal policy (under 300 words) for a [TYPE] agency instituting a mandatory review log before any AI-assisted deliverable ships to a client.

Requirements:
- Every AI-touched deliverable type has one named accountable reviewer, not a team
- The reviewer logs what they specifically checked (facts, numbers, claims, tone, client-specific context) before the deliverable is marked ready to send
- Make clear this replaces "the team reviewed it" as an acceptable status, not adds a second check on top of it
- Keep it short enough that nobody can claim they didn't read it
3

Run a quiet audit to find out where complacency is already happening

Before rolling the new system out, find out how bad the gap already is. Pull a sample of recent AI-assisted deliverables and the Slack or Notion threads around them, and check whether a specific person can be named as having reviewed each one, or whether the record only shows 'looks good' with no indication anyone checked anything in particular.

Here are threads and notes from [NUMBER] recent AI-assisted deliverables:

[PASTE THREADS/NOTES]

For each one, flag whether there's evidence a specific named person reviewed the actual content (facts, numbers, client-specific details) versus just approved that it looked finished. Separate "real check happened" from "assumed fine" — don't be generous with the first category.
4

Scale review depth to risk, not to how the deliverable looks

AI output all looks equally polished whether it got five minutes or an hour of attention, which is exactly why a flat review standard fails. Set two tiers: deliverables with client-facing numbers, claims, or legal/financial detail get a named reviewer doing a real line check; lower-stakes internal drafts get a lighter named spot-check. Either way, a specific person's name is attached to the outcome.

5

Audit the review log monthly for drift, not just existence

A log that exists isn't the same as a log that's working. Once a month, sample a handful of logged reviews and ask whether the notes describe something specific that was caught or fixed, versus the same generic 'looks good' entry repeated on autopilot. A log full of rubber stamps means the accountability moved on paper but not in practice, and it's worth knowing that before a client finds the gap first.

What changes

A workflow where every AI-touched deliverable has exactly one named person accountable for catching problems before a client does, a visible log that proves the check actually happened instead of assuming it did, and an early read on where complacency is quietly setting in before it turns into a client-facing mistake.

The assumption was that AI complacency — skipping the check on AI output even when it's wrong — would come down to one of two things: people trusting the AI too much, or not having the skill to catch the mistake in the first place.

A recent study spanning six experiments and more than 1,300 participants, including 160 service workers, tested that assumption directly. It didn't hold up. The strongest driver of AI complacency wasn't overconfidence in the tool and it wasn't a skills gap. It was the absence of clear accountability for monitoring — nobody had explicitly made checking the output someone's job, so nobody did it as if it were.

The uncomfortable part for agencies isn't that this happens. It's how closely it maps to how most shops already run AI-assisted work — a draft goes out, "the team" reviews it, and it ships, with no single person who was actually on the hook for catching the mistake.

"The team reviewed it" isn't an answer

Ask most agencies who specifically checked the numbers in last week's AI-assisted client report, and the honest answer is usually a shrug toward a department, not a name. That's not a minor process gap — it's precisely the condition the research identifies as the strongest predictor of complacency. When responsibility is shared across a team instead of assigned to a person, everyone can reasonably assume someone else already looked, and the step that was supposed to catch the error quietly doesn't happen for anyone.

AI makes the gap invisible until it isn't

The reason this is easy to miss internally is that AI output renders uniformly polished regardless of how much real attention it got. A report that got a genuine fact-check and a report that got a glance both look equally finished sitting in the same Google Doc. There's no visual signal of "this one wasn't actually checked" until a client spots the wrong number or the outdated claim first — at which point the agency's own review process is the thing that gets questioned, not just the AI.

Fix the structure, not the intentions

This isn't a motivation problem, so a reminder in a team meeting won't fix it. It's a structure problem, and the fix is structural: name one specific person accountable for each deliverable type, require a logged note on what they actually checked before anything ships, and treat "the team reviewed it" as no longer an acceptable status. The point isn't to add more review overall — it's to make sure the review that's supposed to be happening is actually happening, attached to someone who knows it's theirs to catch.

Watch the log, not just whether it exists

A review log can fail quietly too, by filling up with the same generic "looks good" entry on every line. A monthly spot-check of the log itself — does it show evidence of something specific being checked, or just a box being ticked — is what separates a system that's actually working from one that moved the appearance of accountability without moving the substance of it.

Bottom line

AI didn't create a judgment problem at most agencies. It created an accountability problem that was always there, now with higher stakes, because the output looks finished whether or not anyone actually checked it. The fix isn't more trust in the tool or more training on when to doubt it — it's making sure there's always exactly one named person whose job it was to look, and a record that proves they did.

More agency plays every week.

Real workflows for agency founders, not generic AI advice.

Subscribe