Ploy's Real Numbers on Switching from Claude Opus 4.8 to GPT-5.6 Point to a Billable AI Agency Service: Model Migration Audits.

Ploy did something most companies only do internally: it published the exact numbers behind swapping the model underneath a production AI agent.

That is a gift to anyone who wants to sell AI migration work.

What Ploy actually reported

Ploy's agent "plans a page, reads the codebase, writes components, generates imagery, screenshots its own work, and decides when it's done." For four months it ran on Claude Opus 4.8 as the default, and the team says it "tested every frontier release" without finding anything that beat it.

Then GPT-5.6 Sol shipped, and the comparison changed:

• Mean cost per completed build: $3.06 (Opus 4.8) versus $2.22 (GPT-5.6 Sol) — 27% cheaper
• Wall-clock time: 8 minutes (Opus 4.8) versus 3 minutes 42 seconds (GPT-5.6 Sol) — 2.2x faster
• Input tokens: 2.60M versus 1.70M
• Output tokens: 33.0K versus 17.1K
• Visual score: 0.936 versus 0.970

Ploy summarized the case for switching as "builds finishing in less than half the wall-clock time, at 27% lower cost, scoring at or above our incumbent on completed work."

The business idea

Most companies running production AI agents do not have a benchmark harness like Ploy's. They pick a model, ship it, and then keep paying its bill without re-testing every time a new frontier model claims to be faster or cheaper.

That gap is the service.

You are not selling "AI strategy." You are selling a model migration audit: take the client's own recurring agent task, run it against their current model and one or two challengers, and report cost per completed task, wall-clock time, and a quality score — the same shape of comparison Ploy just published for free.

The technical moat is in the failure modes, not the win

Ploy's writeup is valuable precisely because it does not stop at the good numbers. It documents what broke:

• Tool call schema issue: GPT-5.6 sends all 25 parameters on every call versus Claude's selective approach, and the model invented values for unused parameters like offset: 0, making them indistinguishable from intentional arguments. This caused "52-64% of file reads to return empty results." The fix was rewriting optional properties as "required but nullable" using anyOf: [T, null] at the provider boundary.
• Prompt caching redesign: GPT-5.6 eliminated partial-prefix matching and requires explicit prompt_cache_breakpoint markers plus mandatory keys. Ploy says OpenAI's cache nodes "sustain approximately 15 requests per minute before traffic fans to independent cold caches." After implementing workspace-scoped keys, Ploy went from roughly 0% first-call cache hits to 83.7%.
• Reasoning replay: Ploy had to set store: false to stop "Item 'rs_...' not found" errors, requesting encrypted reasoning content instead of server-side pointers.

Anyone can quote a cost-per-build number from a blog post. Very few people have actually debugged a 52-64% empty-read rate down to a schema fix. That is the part clients are paying you to have already solved.

How to package the offer

1. Migration readiness audit

A short paid engagement. Map the client's current model, agent tools, cost per task, and where a schema or caching mismatch is likely to break on a provider switch.

2. Benchmark sprint

Run the client's real recurring task against their current model and one challenger. Report cost per completed task, wall-clock time, and a quality score, mirroring Ploy's own before/after format.

3. Migration execution

If the numbers justify the switch, do the schema, caching, and reasoning-replay rework — the exact category of fix Ploy documented — so the client doesn't discover a 52-64% empty-read bug in production.

4. Standing re-benchmark retainer

Every frontier release is a reason to re-run the comparison. Ploy's team did this internally for four months before GPT-5.6 cleared the bar; most clients would rather pay someone else to keep watching.

Bottom line

Ploy didn't just improve its agent. It published a template for the pitch: name the exact cost and speed delta, name the exact technical breakage, and name the exact fix. Package that same rigor as an audit service and the sales conversation is already half-written for you.

Sources:

https://ploy.ai/blog/migrating-a-production-ai-agent-to-gpt-5-6

https://news.ycombinator.com/item?id=48882716