Showdown Thoughts: The Three-Pass Pattern

Model Showdown Round 5 ended with a leaderboard. Sonnet 4.6 won on the rubric. Opus 4.7 placed second. Qwen 3.5 contributed almost nothing structural. That's the measurement story.

This is the methodology story — what happened after the scores were revealed.

The Problem With Picking a Winner

The naive workflow after a bakeoff is: pick the best run, merge it to main, ship it. Winner takes all.

That's wrong, and Round 5 made it obvious why.

The winning run (Sonnet 4.6) had the best overall rubric score. It also had a weaker path validator than Opus 4.7, and its orphan-matching logic would have missed real-world cases that Opus 4.6 caught. The second-place run (Opus 4.7) had the best validator and the cleanest route structure, but the worst data source choice — reading from the build-time filesystem instead of the live GitHub Contents API.

No individual run was what I'd ship. Each one had at least one bad call. The bakeoff's real output wasn't a winner. It was a map.

When 4 of 4 models made the same design choice, that choice was obviously right. When they diverged — on validation strictness, on data source, on UX for destructive actions — that divergence was the signal. Those were the actual design decisions, the ones worth spending judgment on.

The Three Passes

What emerged from Round 5 is a pattern I've now run twice and would reach for again on any feature where the design space is unclear:

Pass 1 — Bakeoff. Run N models (I used 4) on the same prompt in isolated sessions. Judge blind, before you know which branch is which. Score against a rubric. The output of this pass isn't any of the N implementations — it's the decision map. You now know which choices are contested and which are obvious.

Pass 2 — Merge. Write down a merge plan before touching any code: for each contested layer, which run's approach wins and why. Then ask an agent to compose the merged best-of from those inputs. The merge is strictly better than any individual bakeoff run because it draws on information none of the bakeoff contestants had — the scored comparison of all four.

For Round 5 the plan looked like this:

Qwen 3.5 contributed nothing structural to this table. The bakeoff said "skip this one" clearly enough that there was nothing to debate. That's useful information too — knowing which pieces to skip is part of the map.

The merge was 13 files changed, +990/-9. One TypeScript error caught and fixed. Build passed first try after that. Opened as a PR with the heritage table in the description so future reviewers can trace any decision back to its source run.

Pass 3 — Polish. The merged feature went live. I opened it against real production data and spotted four things immediately: truncated directory names with no tooltip, delete buttons invisible on touch devices, no bulk delete UI despite the API supporting paths: [], and an orphaned section header that would show with count 0 after the lone orphan was deleted.

None of those were predictable before live use. You can't predict friction from a code review — you observe it. The polish pass had to come after the merge because the artifact it was polishing didn't exist until then.

The polish was 6 files changed, +265/-54 and about 20 minutes of agent time.

When to Use It

The pattern has a real cost: the bakeoff is N full agent sessions, each producing a complete implementation that you won't ship. For Round 5 that was ~$35 in inference and a few hours of judging.

That's cheap insurance when the feature has any of these properties:

• Destructive verbs. Delete, update, payment, permission change. The cost of getting validation wrong outweighs the cost of the bakeoff.
• Multiple defensible architectures. Where should validation live? What's the data source? How does auth thread through? When you genuinely don't know the right answer, a bakeoff shows you the option space.
• Hard to change later. Database schemas. Public API contracts. Anything that will accumulate callers.

It's overkill for a 20-line UI tweak or a feature with a single obvious implementation. The signal value of the bakeoff scales with how uncertain you are about the design.

What I'd Do Differently

Three things I'd change for the next run:

Name the contestant chats before pasting the prompt. All four Round 5 chats showed up as "New Chat" in the Coder API cost summary, which meant 20 minutes of token-volume detective work to figure out which cost belonged to which run. Five seconds of effort would have prevented that.

Capture per-phase stats. I have clean bakeoff numbers. I don't have separate merge or polish numbers — they're folded into the judging thread. A lightweight wrapper script around each phase would make the next iteration measurable end-to-end.

Write the polish friction items down before fixing them. I noticed four issues and fixed them in one pass, which collapsed the "observed" list and the "fixed" list into the same moment. Separating them — even by five minutes — would have made the "what does live-review surface" lesson sharper for the writeup. And occasionally you'll notice something that isn't worth fixing.

By the Numbers

• 3 phases: Bakeoff (4 parallel attempts), Merge (1 informed pass), Polish (1 live-review pass)
• 4 implementations produced in the bakeoff, 0 shipped to main as-is
• 3 of 4 bakeoff runs contributed at least one structural piece to the merge
• 13 files changed in the merge pass (+990/-9)
• 6 files changed in the polish pass (+265/-54)
• 4 friction items caught in polish that couldn't have been predicted before live use
• ~$35.56 inference cost for the bakeoff phase
• ~45 min bakeoff (parallel), ~30 min merge, ~20 min polish