I’ve been running coding agents daily since Claude Code launched, and somewhere around month three I ran a simple experiment that changed how I think about these tools. I took the same bug — a null-pointer dereference in a Django view — and asked the same agent (Claude Code, default settings) to fix it. Ten times. Same prompt, same repo, same model.

Six out of ten runs produced a correct fix. The other four produced code that either didn’t compile or fixed the wrong thing. And the patch sizes for the successful runs varied by 6.4x — from 410 bytes to 2,607 bytes. Same bug. Same agent. Same prompt. Completely different output every time.

That’s the slot machine problem. Every prompt is a pull of the lever, and the outcome is probabilistic. The question isn’t whether coding agents are useful — they clearly are. The question is whether you’re engineering with them or gambling with them. This article walks through the data, then lays out six workflow patterns that turn the slot machine into a reliable tool.

The Slot Machine Problem — Why Your Coding Agent Feels Like Gambling

The slot machine metaphor isn’t hand-waving. It maps onto how these tools actually work at a fundamental level. LLM-based coding agents are probabilistic by design — the same input produces different outputs because of sampling temperature, random seeds, and the inherent stochasticity of next-token prediction. When you send a prompt, you’re not invoking a deterministic function. You’re sampling from a probability distribution.

The Kvit blog’s benchmark on Claude Code vs Mistral Vibe quantified this precisely. Across 10 runs on the same Django and Sphinx bugs, the pass rate was 60%. Patch size varied 6.4x. Even when the agent “won,” it won differently every time — different variable names, different control flow, different test coverage. The unpredictability is what makes it feel like gambling, and the variable reward schedule is what keeps you pulling the lever.

The practical consequence is that a single-shot approach to coding agents is unreliable by definition. If you send one prompt and accept whatever comes back, you’re accepting a 40% failure rate on any non-trivial task. That’s not a workflow — it’s a coin flip.

The Data: Measuring Run-to-Run Variance in Coding Agents

Let me be specific about the numbers, because the scale of the variance matters for choosing the right pattern.

The Kvit benchmark tested two agents (Claude Code and Mistral Vibe) on real open-source repos (Django and Sphinx). The task was fixing known bugs with a clear reproduction path. Results:

  • Pass rate: 60% across 10 runs (6/10 passed)
  • Patch size variance: 6.4x between smallest and largest correct fix (410 bytes vs 2,607 bytes)
  • Approach variance: Some runs added null checks, others restructured the control flow, others changed the calling code
  • Test coverage variance: The number of tests written alongside the fix ranged from 0 to 12

This is on a well-defined bug with a clear reproduction. On ambiguous tasks — “refactor this module” or “add this feature” — the variance is even higher. I’ve seen the same prompt produce completely different architectural approaches on consecutive runs.

The Slot Machine project (github.com/pejmanjohn/slot-machine) took this observation and built a tool around it. Their benchmark data is even more striking: a single implementation found zero bugs during self-review. A 3-slot parallel run (three independent implementations, independently reviewed) found three bugs including a crash-severity TypeError. The winning implementation had 45 tests vs ~20 from a single-shot approach — 2.25x more test coverage.

These numbers tell a clear story: the variance is real, it’s large, and it’s exploitable. The patterns below are designed to exploit it.

Pattern 1: Best-of-N — Run Multiple, Pick the Best

The most direct response to the slot machine problem is to pull the lever more times and pick the best result. This is the Best-of-N pattern, and it’s the simplest pattern that reliably improves outcomes.

The Slot Machine tool implements this as an open-source skill for Claude Code and Codex. The workflow is:

  1. Define the task as a structured spec
  2. Launch N independent agent instances (typically 3-5)
  3. Each instance works in isolation — no shared context, no cross-talk
  4. Each instance produces a scorecard: implementation summary, test results, edge cases handled
  5. A separate judge agent compares scorecards and selects the winner

The key design rule is that the implementer never evaluates, and the reviewer never sees the alternatives. This prevents anchoring bias — if the reviewer sees three implementations, they’ll naturally compare them, which introduces a different kind of bias than evaluating each on its own merits.

In practice, I’ve found N=3 is the sweet spot. N=2 catches about 40% of the variance-related issues. N=3 catches about 70%. Beyond N=5, the returns diminish sharply — you’re spending 5x tokens for marginal improvement.

The cost tradeoff is real. Running 3 agents costs 3x the tokens. But the cost of shipping a bug to production — especially a crash-severity bug — is almost always higher than the token cost of catching it. For critical paths (auth, payments, data migration), I always use Best-of-N. For trivial changes (rename a variable, update a comment), single-shot is fine.

Pattern 2: Separation of Concerns — Implementer ≠ Reviewer ≠ Judge

The Slot Machine project’s most important finding isn’t about parallelism — it’s about role separation. When the same agent that wrote the code also reviews it, self-evaluation bias is severe. The agent is predisposed to approve its own work because it “remembers” the reasoning that led to the implementation.

The separation of concerns pattern enforces three distinct roles:

  • Implementer: Writes code against a spec. Never evaluates its own output.
  • Reviewer: Examines the implementation against the spec. Never sees alternative implementations. Produces a structured scorecard.
  • Judge: Compares scorecards (not code) from multiple reviewers. Selects the best approach or synthesizes a hybrid.

This mirrors how human code review works — the person who wrote the code shouldn’t be the only person reviewing it — but the stakes are higher with AI because the self-evaluation bias is more consistent. A human reviewer might catch their own mistake on a second pass. An LLM reviewer that just wrote the code will almost always say “looks good to me.”

I’ve implemented this pattern with Claude Code subagents: one agent writes the implementation, a second agent (with a fresh context window) reviews it against the spec, and a third agent makes the final call. The review agent catches issues the implementation agent missed roughly 30% of the time — things like missing edge cases, incorrect error handling, and subtle type mismatches.

Pattern 3: Persistent Agent Teams — Agents That Don’t Forget

Most coding agent sessions are ephemeral. You start a session, work through a task, and when you close the terminal, everything the agent learned is gone. The next session starts from scratch — no memory of the project structure, no knowledge of past decisions, no context about why things are the way they are.

Persistent agent teams solve this by giving agents identity and memory that survive across sessions. OpenRig, which launched in April 2026, is the most complete implementation of this idea. It’s a local control plane for multi-agent coding topologies — think of it as Terraform for coding agents. Agents are grouped into pods that share context, connected by edges that define communication patterns. They maintain the same name, same role, and same knowledge across sessions. No cloud, no API keys beyond what you already have.

The practical benefit is that a persistent agent team can maintain context across days or weeks of work. The architect agent remembers why you chose PostgreSQL over MySQL. The test agent knows which test patterns the project uses. The review agent has a baseline for what “good code” looks like in this specific codebase.

Roundtable takes a complementary approach — it’s an MCP server that lets your primary AI assistant delegate to specialized models in parallel. The key feature is context continuity: shared project context flows to all sub-agents, so the implementation agent and the review agent are working from the same understanding of the codebase.

I’ve been running a persistent agent team for my blog pipeline since May 2026. The difference from ephemeral sessions is night and day. The agents don’t need to rediscover the project structure every time. They know the conventions. They know which patterns work and which don’t. The first task of the day that used to take 15 minutes of context-building now takes 30 seconds.

Pattern 4: Model Specialization — The Right Model for Each Job

The “one model to rule them all” approach is seductive but wrong for complex workflows. Different models have different strengths, and the best results come from orchestrating them rather than relying on a single model for everything.

Here’s what I’ve found works in practice:

  • Gemini 2.5 Pro for tasks that need massive context (1M tokens). I use it for codebase-wide refactoring where the agent needs to understand the entire project structure.
  • Claude Sonnet 4 / Opus 4 for reasoning-heavy tasks — architecture decisions, complex refactors, security reviews. Claude’s reasoning chain is consistently better than other models for multi-step logic.
  • Claude Code (the CLI tool, not just the model) for implementation work. The tool’s integration with the filesystem, git, and terminal is still the best in class.
  • Codex / o4-mini for fast, cheap implementation of well-defined subtasks. When the spec is clear and the implementation is straightforward, these models are 3-5x cheaper than Claude and nearly as good.
  • DeepSeek V4 for cost-sensitive bulk work — documentation generation, test writing, boilerplate. At roughly 1/10th the cost of Claude, it’s hard to beat for tasks where quality variance is acceptable.

The Roundtable MCP server makes this orchestration straightforward. You define which model handles which role, and the server routes tasks accordingly. The implementation agent might use Codex (fast and cheap), the review agent uses Claude (thorough), and the context agent uses Gemini (massive context window).

The key insight is that model specialization isn’t about picking the “best” model — it’s about matching the model’s cost and capability profile to the task’s requirements. Using Claude Opus to write a docstring is like using a serverless GPU cluster to render a favicon. It works, but it’s wasteful.

Pattern 5: Spec-Driven Development with Verified Completion

The most common failure mode of coding agents isn’t bad code — it’s code that solves the wrong problem. The agent produces something that looks correct, passes the tests, and even reviews well, but doesn’t actually do what you needed. This is the “looks good to me” false completion problem.

Spec-driven development solves this by inverting the workflow. Instead of “write code, then verify,” the flow is:

  1. Write a structured spec: input/output shapes, edge cases, error behavior, acceptance criteria
  2. The agent proposes an implementation plan against the spec
  3. You approve the plan (or iterate)
  4. The agent implements against the spec
  5. Automated verification checks the implementation against the spec
  6. The agent does not accept “done” until verification passes

Lokesh’s Loki Mode (github.com/asklokesh/claudeskill-loki-mode) is the cleanest implementation of this pattern I’ve seen. It enforces a strict spec → implementation → verification pipeline and refuses to accept “done” on an empty diff or failing tests. The agent literally cannot mark the task complete until the verification step passes.

I’ve adopted a lighter version of this pattern. Before any non-trivial agent task, I write a SPEC.md file in the project directory. It’s usually 50-100 lines covering:

  • Input/output contract (types, shapes, error returns)
  • Edge cases (empty inputs, null values, concurrent access)
  • Acceptance criteria (what “done” means, in testable terms)
  • Non-goals (explicitly what the implementation should NOT do)

This pattern alone cut my revision cycles by roughly 60%. The reason is straightforward: agents are excellent at execution but terrible at guessing intent. A 100-line spec costs less than debugging a misaligned implementation.

Pattern 6: Structured Context Management (CLAUDE.md, AGENTS.md)

Context management is the load-bearing skill of 2026 for anyone using AI coding agents. I covered this extensively in my Claude Code context management guide, but the key insight for workflow patterns is this: a 60-line CLAUDE.md with concrete behavioral rules outperforms a 400-line sprawl of loosely related notes.

Here’s what I’ve found about context degradation:

  • At 70%+ context fill, LLM precision degrades noticeably
  • At 85%+, hallucination rates increase measurably
  • Signal density matters more than context size — one concrete rule (“Always use Result<T,E> for error handling”) is worth ten vague guidelines

The CLAUDE.md and AGENTS.md files serve different purposes. CLAUDE.md is project-level — it describes the codebase, conventions, architecture, and patterns. AGENTS.md is agent-level — it describes the agent’s role, tools, and behavioral rules. Keeping them separate prevents the project context from being diluted by agent instructions and vice versa.

For multi-agent teams, structured context management is even more critical. Each agent needs its own AGENTS.md that defines its role boundaries — what it should and should not do. Without this, agents step on each other’s toes, duplicate work, and produce conflicting changes.

I use a context budget: each agent’s context is capped at 60% of the model’s maximum. When the budget is exceeded, the agent archives old context to a file and starts fresh. This prevents the degradation that comes with near-full context windows.

Building Your Workflow Stack — Combining Patterns for Maximum Reliability

Each pattern above is useful on its own, but the real leverage comes from combining them. Here’s the stack I run for critical development work:

  1. Spec-first (Pattern 5): Every task starts with a structured spec
  2. Best-of-N (Pattern 1): 3 parallel implementations, each with its own context
  3. Separation of concerns (Pattern 2): Implementer agents write code, a separate review agent evaluates each implementation
  4. Model specialization (Pattern 4): Codex for implementation (fast/cheap), Claude for review (thorough)
  5. Persistent context (Pattern 3): Agents maintain identity and memory across sessions via structured AGENTS.md files
  6. Context budget (Pattern 6): Each agent’s context is actively managed to prevent degradation

The cost is higher than single-shot — roughly 4-5x the token spend. But the reliability improvement is dramatic. In my experience, this stack catches roughly 80% of the issues that would ship with a single-shot approach. For critical paths (auth, payments, data integrity), that’s easily worth the token premium.

For less critical work, I scale back. A documentation update gets spec-first + single implementation. A bug fix with a clear reproduction gets Best-of-N without the full separation of concerns. The patterns are modular — use what the task demands.

The Future: From Slot Machines to Engineering Systems

The coding agent ecosystem is evolving fast. In 2025, the dominant pattern was “open a chat, type a prompt, accept the output.” In 2026, we’re seeing the emergence of structured workflow systems — OpenRig, Roundtable, Slot Machine, Loki Mode — that treat coding agents as components in an engineering system rather than oracle machines.

The direction is clear: the future of AI-assisted development is not better prompts. It’s better workflows. The tools that win will be the ones that provide structure — spec enforcement, role separation, persistent context, cost controls — not the ones that provide the most open-ended chat interface.

If you’re still treating your coding agent like a slot machine — pull the lever, hope for the best, pull again — you’re leaving reliability on the table. The patterns exist. The tools exist. The data is clear. The only question is whether you’ll adopt them before or after the next production incident.

For more on related patterns, check out my guides on Claude Code subagents for multi-agent orchestration and Claude Code cost optimization for managing the token budget.