This review is deliberately narrow. We recap the product in one paragraph and then spend the rest of the article on the three things no catalog page does: compute the error bar, price the trade-off, and test the claim against category-level evidence. If you want the feature tour, see our earlier Autoprompt walkthrough.

What Is the Autoprompt Skill in One Paragraph?

Autoprompt is a coordination layer, not another coding agent. It installs into a host tool — Claude Code, Codex, OpenCode and eight others — and, when explicitly invoked, runs a fixed loop: choose a route, plan, build, check, independently verify, then finish. Roles are split across levels L0 to L4 so that no single agent plans a change, approves it and certifies its own verification. It requires Node.js 20+, Python 3.11+ with PyYAML and Bash 4.3+, and v2.0.0 ships a single CLI that activates the skill per provider. That is the whole product; everything below is about whether its number survives scrutiny.

What Does “45% Fewer Failures” Actually Measure?

It measures one metric on one benchmark, once. Baseline OpenCode 1.18.7 solved 60 of 89 Terminal-Bench 2.1 tasks. With Autoprompt active, the same harness on the same model solved 73 of 89. Failures fell from 29 to 16.

That is a defensible statement, and it is also the limit of what the published evidence supports. The figure is a reduction in the failure rate on a single 89-task run, not a reduction in defects you will see in your repository, and not a comparison against any other tool.

Every row is the same data. The “45%” headline is the most flattering row that is still arithmetically true, which is exactly why it propagated and the rest did not.

Why Is a 45% Reduction Really a 1.81x Claim?

Because a percentage reduction in failures is not a percentage improvement in capability, and the difference is large enough to change how you budget for it.

Two useful translations exist. In odds terms, 29 failures becoming 16 is 1.81x fewer failures — the treatment arm fails about 55% as often. In rate terms, 32.58% becoming 17.98% is a 44.8% reduction in the failure rate, rounded to 45%. Both are honest. “45% better agent” is not, and it is the phrase most readers hear.

For a working developer the odds reading is the operative one: on a hard task where the plain agent has a 33% chance of failing, the wrapped agent has roughly an 18% chance, so you still expect to repair one task in six. That is a meaningful improvement and it is not a transformation.

What Is the Confidence Interval on the 45% Claim?

The error bar nobody publishes is +2.02pp to +27.20pp at 95% confidence. That is the single most useful number in this review.

Read the interval, not the point estimate. The result is statistically significant — the lower bound clears zero — but the CI spans from “a marginal benefit you might not notice” to “a near doubling of the failure reduction you were promised.” A 14.61-point change on 89 tasks cannot distinguish those two worlds, and it cannot rule out an effect as small as two points.

The lower bound is worth translating. At +2.02pp, the wrapped agent would pass 69.44% instead of 67.42% — about 61.8 solves instead of 60, meaning roughly one extra task solved out of 89 for the same ~2x token spend. That is the pessimistic-but-consistent version of the claim. Nobody selling this skill prints it.

The power figure matters too. With n=89, a study has only about a 61% chance of detecting an effect the size of the one actually observed. This is a single run reporting a positive result, not a replicate — and single runs in this size class routinely produce point estimates that shrink on retest.

Is the Evidence Asymmetric — Did the Treatment Arm Keep Its Receipts?

No. The baseline retained all 89 per-task verdicts; the Autoprompt arm did not, and that asymmetry is more serious than a dead link.

The benchmark document links to a per-task verdict file that returns HTTP 404, and the top-level benchmark/ directory it lived in is absent from the repository’s main branch — verified by walking the full git tree on 2026-10-01, not inferred from the broken link. The document itself concedes that the Autoprompt arm “cannot be rebuilt task by task” because “its original per-task map was not retained.” The baseline’s ledger exists; the treatment arm’s does not.

This inverts the usual storage pattern. When a benchmark claims an improvement, the arm whose result is surprising is the arm whose raw output you most need to inspect — to check for task contamination, harness leakage, retries, or selective reporting. Here the ordinary arm is fully auditable and the extraordinary one is a summary table.

None of that is proof of error, and it should not be read as an accusation. It is a statement about what a reader can verify: the published counts can be re-analysed (the interval above is that analysis), but the underlying per-task result cannot be reproduced or falsified without rerunning the whole benchmark at your own expense. A result you cannot rebuild is a result you must trust, and trust is the one input a benchmark is supposed to remove.

Does the v2 Release Invalidate the Benchmark?

For the claim as currently marketed, yes — the 45% is version 1 evidence attached to a version 2 product.

v2.0.0 shipped on 2026-09-09 with a new CLI activation path, eleven provider adapters, path= controls (auto, direct, light, roadmap) and new concurrency controls. The benchmark was run with OpenCode 1.18.7 in the v1 line; the current repository lists OpenCode 1.18.29 among its tested hosts. As of 2026-10-01 the docs/benchmarks/ directory contains only the Terminal-Bench 2.1 file, a Codex canary note and a low-compute mechanism note — there is no v2 benchmark.

That distinction is not pedantry. A new activation path, new routing controls and eleven adapter changes alter the very execution path the benchmark measured. Every article published after 2026-09-09 that presents 45% as the current, measured performance of the software you are about to install is citing a superseded artifact. The honest framing is: “v1 measured +14.61pp on one run; v2 is unmeasured.”

How Does Autoprompt Compare to Other Agent Skills?

Autoprompt sits in a small minority that works at all, and about in the middle of the curated-skill field — and the second half of that sentence is the part nobody likes.

Two independent benchmarks bracket this category. SWE-Skills-Bench evaluated 49 skills across roughly 565 task instances with deterministic pytest verification and found that 39 of 49 skills produced zero pass-rate improvement, with an average gain of just +1.2%. Token overhead ranged from −78% to +451% while pass rates stayed flat, and three skills actively degraded performance by up to −10%. Anyone assuming a popular skill helps has roughly an 80% chance of being wrong about a random one.

SkillsBench is the kinder baseline: 87 tasks across 8 domains and 18 model-harness configurations, with matched no-skill and with-skill conditions. Curated skills lifted average pass rate from 33.9% to 50.5% — +16.6pp, a 25.5% normalised gain, with per-configuration gains from +4.1pp to +25.7pp.

Set Autoprompt’s +14.61pp against those numbers:

The headline that looks enormous in isolation is slightly below the fleet average of curated skills in context. Both statements are true, and together they give the honest verdict: Autoprompt is in the minority of skills that measurably help, and it is not an outlier among them. If your baseline assumption was “agent skills do nothing” (a 1.2% expectation), this is a large win. If your baseline was “curated skills give about 16 points,” this is ordinary.

What Is the Trap Next Door for Self-Generated Skills?

The same SkillsBench work contains the finding most relevant to anyone tempted to copy the pattern: when agents were asked to author their own skills before starting a task, they performed worse than baseline, dropping 8.1 to 11.5 percentage points.

That is a direct warning for teams that read Autoprompt’s architecture and conclude they can have their orchestrator auto-generate a project-specific checklist each run. The evidence says the opposite. Curated skills help; self-generated skills hurt, plausibly because an agent writing its own procedure optimises for plausibility rather than for the verification steps it is least likely to perform unaided. Autoprompt’s own design leans the right way here — it is explicit, invoked by name (/autoprompt, $autoprompt, /skill:autoprompt) rather than authored on the fly — but the temptation to bolt on auto-generation is real, and the literature is not neutral about it.

What Does the 45% Cost in Tokens and Time?

It costs roughly 3x wall-clock time and 2x tokens, and nobody has published the exchange rate between the two. We can compute an approximate one.

The project’s own README labels the 3x/2x figures “planning estimates based on user experience reports, not measured benchmark results,” which is unusually candid — most projects would print them as fact. But it also means the one number a buyer needs does not exist. At an assumed 1.9x token multiplier, the +14.61pp gain comes at roughly 90% more tokens than baseline; spread across the 13 extra solves, that is about 7% of a full baseline run’s token budget per additional task solved. Or, in the form that survives a business case: you are buying roughly 1.5 percentage points of pass rate for every extra 10% of tokens you spend.

Whether that is worth it depends entirely on the alternative use of the same tokens — a second opinion from a stronger model, a broader test suite, or simply a retry with a different seed. Autoprompt’s honest framing is a price, not a percentage, and the price is currently an estimate of an estimate.

Why Do Failure-Reduction Tools Exist at All?

Because long-horizon reliability, not raw capability, is the binding constraint — and the arithmetic of chained steps explains why a verification layer can help at all.

METR’s time-horizon research found that frontier agents succeed on nearly 100% of tasks a human would finish in under about four minutes, but on under 10% of tasks taking a human over roughly four hours, with the 50%-reliability horizon doubling approximately every seven months since 2019. Capability is not the problem at long horizons; the ability to keep 20 or 50 steps correct is.

Compounding makes that concrete. At 95% per-step reliability, a 20-step task succeeds about 36% of the time and a 50-step task about 8%.

This is why a loop that adds independent verification is a plausible intervention rather than a gimmick: it targets step-level error, which is where the compounding lives. It is also why the size of Autoprompt’s measured gain is plausible — moving an agent from roughly 67% to roughly 82% on a hard terminal benchmark is exactly the scale of improvement you would expect from catching a fraction of mid-trajectory mistakes, not from making the model smarter. And it is why the measurement is hard: the same compounding that makes the intervention valuable makes single-run estimates volatile.

Is 82.02% Actually a Good Score?

It is a mid-table score. Public Terminal-Bench 2.1 leaderboards put top model-harness combinations at roughly 87–91% on the same 89 tasks, so Autoprompt’s 82.02% — with an 89-task, OpenCode 1.18.7, DeepSeek V4 Flash harness — sits below the frontier, not at it.

The comparability warning matters as much as the number. Terminal-Bench scores are not interchangeable across boards: the official board measures agent-plus-model combinations, Artificial Analysis tests models directly with labelled effort tiers, and vals.ai retests everything under a unified Terminus 2 harness. Citing “82.02%” without naming the harness and model invites a wrong read, in either direction — as a disappointing score when the harness was never frontier, or as a frontier score when the board is measuring something else. The safe sentence is the specific one: OpenCode 1.18.7 with DeepSeek V4 Flash and Autoprompt scored 82.02% on Terminal-Bench 2.1.

Why Do the Stars and npm Downloads Point the Other Way?

Adoption is decelerating while the headline number compounds, and the open-issue queue tells you where maintainer attention is going.

The star curve is a launch burst into a slow tail, not compounding growth: roughly 146 stars a day for three days, then about 23 a day for the next five weeks. npm tells the same story more sharply — the published package is still 1.0.4, two minor lines behind GitHub’s v2.0.0, and therefore still carries nine providers and the old mode=/max_subs= interface. Weekly downloads are in the tens.

Meanwhile the three open issues are all Windows activation failures (#27 native Windows refused, #28 OpenCode cmd-shim EINVAL, #29 npm shims for native package bins), and the README’s own requirement line says “these versions passed Linux runs.” The practical reading: the eleven-provider claim is a code-path claim, the verified execution surface is Linux, and if you are on Windows you are currently the maintainer’s queue. If you want the tool to keep working, file issues rather than stars.

How Portable Is Autoprompt Across Its 11 Host Agents?

The adapters are real, and portability is the dimension no agent-skill framework has fully solved.

The tested-version table lists Claude Code 2.1.263, Codex 0.148.0, OpenCode 1.18.29, Kilo Code 7.5.15, VS Code 1.136.1, Prime Agent 0.7.2, Oh My Pi 18.1.14, DeepSeek Harness 0.1.2-rc.1, Reasonix 1.30.0, Hermes Agent 0.21.1 and Grok Build 1.0.13 — eleven hosts, each pinned to a version that passed on Linux. The v2 CLI unifies activation across them, which is genuine progress over the v1 invocation-per-host mess.

Two gaps remain. First, one independent review rates the project’s portability “portable with changes” rather than portable, because the v2 CLI activation path is provider-specific — the unification is at the command surface, not at the execution semantics. Second, custom model routing does not exist on all hosts, so a team that needs to pin a specific model per role will find the coverage uneven. Independent taxonomy work reaches the same conclusion at the framework level: no agent-skill system currently covers specification, context, roles, execution, validation and portability simultaneously. Budget for a portability tax on any host other than Linux plus a mainstream agent.

Who Should Use Autoprompt in 2026, and Who Should Not?

Buy it for hard, ambiguous tasks on a Linux workstation with an established host agent — and skip it everywhere else.

A good fit if you:

• Run Claude Code, Codex or OpenCode on Linux or macOS and already have a working test suite.
• Spend your time on multi-step, underspecified tasks where the agent “gets stuck mid-task” rather than failing instantly.
• Value the verification structure more than the speed — the independent-check discipline is the durable part of the design, and it is transferable even if you later drop the tool.
• Need evidence you can point at: the +14.61pp gain is statistically significant, and you now know its interval.

A poor fit if you:

• Do very small tasks. A 2x token multiplier on a one-file change is pure loss, and the project itself notes gains may vary significantly between small and large tasks.
• Work primarily on Windows today. All three open issues are Windows activation failures.
• Need custom model routing on every host.
• Have no local terminal runtime for the agent to execute in — without that, there is nothing for the verification loop to verify.
• Are choosing between Autoprompt and simply spending the same 2x tokens on a stronger model or a larger test suite. That comparison has never been published, and it is the one that decides most budgets.

Read against the third-party score available — FollowAgents rates the project 69/100, “use with care,” with praise focused specifically on disclosure quality and a caveat that the review is stale on provider count and version — the fair summary is that the engineering and the honesty are both above average while the evidence remains one run.

How Do You Reproduce or Reject the Claim Yourself?

The claim is falsifiable, and the cheapest path is a paired test on your own backlog.

1. Fix the harness. Pin one host agent and one model. Mixed hosts invalidate every comparison, as the three-board divergence on Terminal-Bench 2.1 shows.
2. Select tasks with headroom. Pick 30–50 tasks you already fail. Testing on tasks you pass measures nothing.
3. Run both arms on the same tasks. Baseline run, then an Autoprompt run, same commit, same environment, ideally same day to reduce drift.
4. Score deterministically. Use your test suite or an external grader, not agent self-assessment. This is the step most teams skip, and it is the step that matters — an agent judging its own work is precisely the failure mode the skill exists to prevent.
5. Compute the interval, not just the difference. At 30 paired tasks you have almost no power; a 15-point difference on 30 tasks will not be distinguishable from noise. If your observed difference is small, treat the result as inconclusive rather than negative.
6. Log tokens and wall clock. The project could not, and that is why its cost figure is an estimate. Your run should not repeat that mistake.
7. Run autoprompt doctor --strict before measuring, so a partial install is not scored as a skill failure.

Expect a real but unglamorous outcome. The published effect is +14.61pp with a lower bound of +2.02pp; on a 40-task set, a plausible result is three to six extra solves at roughly double the tokens.

What Is the Verdict on the Autoprompt Skill?

It works, the number is real, and the number is smaller and less certain than the headline implies.

The defensible statement of the evidence is this: on a single 89-task Terminal-Bench 2.1 run, wrapping OpenCode 1.18.7 with Autoprompt raised pass rate from 67.42% to 82.02%, a +14.61pp gain (95% CI +2.02pp to +27.20pp, p=0.025) that reduces the failure rate 44.8% — 1.81x fewer failures — at an estimated 3x time and 2x tokens, with no v2 benchmark yet published.

Everything past the comma is why this article exists. Most surfaces that rank for “autoprompt skill” have already lost it: catalog pages still publish nine providers, v1.0.4 and 940 stars while the repository says eleven providers, v2.0.0 and 1,298 stars. The 45% survived intact while every verifiable detail around it decayed — which is the normal behaviour of a good headline and the reason it deserves an error bar. Use the skill for hard tasks, budget the tokens honestly, and quote the interval.

FAQ

Does the Autoprompt skill actually work? The evidence says yes, within limits. Autoprompt’s own Terminal-Bench 2.1 run moved OpenCode 1.18.7 from 60/89 to 73/89 solves (+14.61pp, p=0.025), and independent category benchmarks show most agent skills produce no measurable gain at all, so a skill that clears the bar is a minority case. However, the result is a single 89-task run, the treatment arm’s per-task ledger was not retained, and the 95% confidence interval spans +2.02pp to +27.20pp. Treat it as a real but imprecisely measured improvement, not a guarantee.

Where does the 45% figure come from and is it accurate? It comes from failures falling from 29 to 16 on 89 Terminal-Bench 2.1 tasks. That is a 44.8% reduction in the failure rate, so 45% is accurate as a description of that specific metric. It is not a 45% improvement in capability: the pass rate rose 14.61 points (67.42% to 82.02%), the relative pass-rate gain is 21.67%, and in odds terms the wrapped agent fails 1.81x less often. Any source describing it as “45% better” has changed the meaning of the number.

What is the confidence interval on the Autoprompt benchmark, and why does it matter? The +14.61pp difference corresponds to a 95% confidence interval of roughly +2.02pp to +27.20pp at n=89 per arm (two-proportion z=2.24; Fisher exact two-sided p=0.038). No competitor publishes this. It matters because the interval is wide: the same data is consistent with a benefit as large as 27 points and as small as 2 points, and at 2 points the tool would win about one extra task out of 89. With n=89, statistical power to detect the observed effect is only about 61%, so this should be read as a positive single run, not a settled effect size.

Does Autoprompt work on Windows, and is the npm package current? Windows is currently the hard gate: all three open issues on the repository as of 2026-10-01 are Windows activation failures (#27 native Windows refused, #28 OpenCode cmd-shim EINVAL, #29 npm shims for native package bins), and the project states that its tested versions “passed Linux runs.” Separately, npm’s latest tag is still 1.0.4 (published 2026-08-21), two minor lines behind the GitHub v2.0.0 release of 2026-09-09, so the npm package still ships nine providers and the old mode=/max_subs= interface. Install from the v2.0.0 release if you need the CLI and the eleven adapters.

How much does Autoprompt cost in tokens and time, and is it worth it? The project estimates roughly 3x wall-clock time and 2x tokens, and explicitly labels both as planning estimates from user experience reports because timing and token logs were not retained. Pair that with the +14.61pp gain and you get approximately 1.5 to 1.6 percentage points of pass rate per extra 10% of tokens — a real but modest exchange rate. It is worth it on hard, multi-step tasks where your agent tends to get stuck and you have a deterministic test suite to verify against. It is a clear loss on small single-file changes, where the multiplier buys nothing.

What the 45% Figure Actually Measures

The number everyone quotes is not a percentage of tasks solved and it is not a percentage reduction in tokens. It is a reduction in failures on one benchmark, measured once.

The run is documented as Terminal-Bench 2.1 with 89 tasks, OpenCode 1.18.7 as the harness, and DeepSeek V4 Flash as the model. Baseline solves were 60 of 89. With Autoprompt active, solves rose to 73 of 89. Failures went from 29 to 16, which is a 44.8% reduction — rounded to 45% in every headline since.

Two things make that table worth reading slowly. First, the denominator: 13 tasks out of 89 is a meaningful but bounded result, and 44.8% is a property of the failure slice (13 of 29), not of the workload. The same result is also described in the project’s own README as “about 2x fewer mistakes,” which is the identical measurement expressed against the smaller surviving set. Neither framing is dishonest; the choice of denominator is what makes a 13-task improvement look like a near-halving.

Second, and more important: cost was not instrumented. The project discloses the ~3x time and ~2x token trade-off as “planning estimates based on user experience reports, not measured benchmark results,” and explains that timing and token logs were not retained. So the accuracy half of the claim is a measured benchmark; the cost half is not a measurement at all. If you are budgeting an unattended run, treat 3x as a floor and verify it on your own task distribution before trusting it in a CI budget.

Autoprompt v2: What Changed Since the Original Review

The repository has moved past the benchmark. Autoprompt shipped v2.0.0 on 2026-09-09, and the README now labels the 45% result as “version 1 benchmarks,” with a note that version 2 benchmarks will follow. No v2 efficacy number exists as of 2026-09-28. That matters for anyone citing the headline today: the marketing line is unchanged, but the artifact it refers to is two releases old.

What v2 actually changed is scope and control, not the accuracy thesis:

The v2 workflow is more explicit about where control sits. Instead of a mode switch, you select a work path — path=auto|direct|light|roadmap — and a concurrency profile. The default is tokensaver, which caps fan-out at six subagents. The documentation is clear that choosing a path does not waive authorization, capability, budget, or verification requirements, and that invalid combinations error out rather than silently falling back to a different route.

The Evidence Boundary Moved, and It Got Thinner

The single most important fact for a 2026 buyer is not in the README: the benchmark evidence is no longer fully checkable.

The per-task verdict file that the benchmark documentation points to for all 89 retained verdicts now returns HTTP 404. The benchmark/ directory that once held the raw artifacts is absent from the main branch — verified by walking the recursive git tree listing of 1,381 paths, not inferred from a link. The methodology summary survives in docs/benchmarks/terminal-bench-2.1.md, but the per-task data a skeptical reader would use to audit the run does not.

That is a reproducibility regression, and it is the correct thing to hold against this project, because prompt-skill vendors have an obvious incentive to publish the summary and not the ledger. A benchmark you cannot audit is a marketing claim with a methodology section. Independent verification does not become impossible — you can rerun Terminal-Bench yourself — but the cheapest possible audit path (read the verdicts) is closed, and that cost falls on the buyer.

The v2.0.0 release notes show a similar honesty pattern in miniature. They concede that “full live acceptance of the final Codex artifact is not established,” and that Windows and macOS builds are installer kits with native runtime verification remaining Linux-only. That is unusually candid disclosure for a release announcement, and it is the reason this review reads the project as careful rather than reckless: it publishes caveats that hurt it. It simply has not published version 2 evidence yet.

Eleven Providers, One Verified Platform

Provider count is the feature most third-party articles get stale. Autoprompt now claims 11 supported hosts. The v2.0.0 work extended the Codex workflow across all of them and added Hermes Agent and Grok Build to a list that previously stopped at Codex, OpenCode, and Prime Agent.

The platform support story is narrower than the provider story. All three open items on the repository concern Windows:

• Issue #27 — activation is refused on Windows with PROVIDER_UNSUPPORTED because all 10 reviewed local records are linux/x64 and trusted-public-keys.json lists no Windows keys.
• Issue #28 — OpenCode activation fails on the npm cmd shim.
• PR #29 — an in-flight fix for Windows npm shims for native package bins.

Read together with the release notes’ Linux-only verification line, the practical rule is simple: the 11-provider claim is a code-path claim, and the verified execution surface is Linux. If your team codes on Windows, this is not yet a drop-in tool, and the open issues say so more precisely than any review can.

Paths and Run Controls: Where v2 Spends Your Tokens

Cost in this category is not a fixed multiplier; it is a budget you set. The mechanism is straightforward and is the same one the independent literature describes: every phase is another model call over the same code, the judge is a separate model call with its own context, and parallel lanes re-read the same files. Fan-out is therefore the dominant cost lever, more than the model choice per phase.

The v2 controls map to that reality directly:

• --concurrency tokensaver — at most six subagents, the default. Sensible for a laptop or a metered API key.
• --concurrency wide — maximum parallel lanes. Fast, and the most expensive way to run it.
• --concurrency custom --max-subs N — the setting most teams should actually use, because it lets you align fan-out with the size of the change rather than the optimism of the moment.
• path=direct — minimal routing for a small, well-specified change.
• path=light / roadmap — progressive planning depth for larger or ambiguity-heavy work.

One operational caution worth repeating from independent walkthroughs: six lanes can touch deployment credentials, so run unattended sessions in a disposable VM or a container with scoped keys rather than on your working machine. And there is a real quality floor below which orchestration is pure overhead — if the task has nothing to execute or test, the verification phase produces one more opinion at full token price.

The default tokensaver profile is a reasonable acknowledgment of that floor. It is also the setting to keep when you are running agents on a VPS budget, where a wide fan-out can turn a ten-minute change into a twenty-dollar one.

What Independent Skill Benchmarks Say About This Category

Autoprompt is one instance of a category that has been measured independently, and the category averages are not kind. Any review that quotes 45% without quoting the surrounding literature is selling, not informing.

The closest comparable study, SWE-Skills-Bench, ran 565 task instances across 49 public SWE skills with GitHub repositories pinned at fixed commits and execution-based verification. Thirty-nine of the 49 skills produced zero pass-rate improvement. The average gain was +1.2%. Token overhead ranged from -78% to +451% while pass rates stayed flat. Seven specialized skills gained up to +30%, and three actively degraded performance by up to -10% through version-mismatched guidance.

SkillsBench, testing 87 tasks across 18 model-harness configurations with matched no-skill controls, found that curated skills raise the average pass rate from 33.9% to 50.5% — a +16.6 percentage point gain — but with gains ranging from +4.1 to +25.7 points, and 13 of 87 tasks showing negative deltas. Its most useful structural finding is that skill value is a property of the specific stack, not of skills in general. Compact focused skills beat exhaustive bundles by a wide margin, and loading four or more skills together yielded only +10.1 points.

A third line of work explains the failure mode that matters most for a verification-heavy tool. The paper “Agent Skills Can Be Harmful” confirmed 307 skill-induced failures across those same benchmarks: 125 functional and 182 efficiency regressions. Efficiency damage was dominated by what the authors call Excessive Procedure at 62.6% — not prompt length — with excessive verification (67 cases) and heavy implementation pipelines (30 cases) as the largest drivers. Functional failures rarely came from irrelevant skills; seemingly relevant skills caused wrong or omitted implementation elements in 68.8% of cases. That is Autoprompt’s exact design surface: it turns verification checklists into mandatory work, which is simultaneously the mechanism of its accuracy claim and the largest documented source of token regression in the literature.

Two further results bound the realistic ceiling. Under progressively realistic conditions, where the agent must retrieve skills itself instead of receiving a hand-picked one, skill benefits decay toward no-skill baselines — and query-specific refinement recovers performance, lifting Claude Opus 4.6 from 57.7% to 65.5% on Terminal-Bench 2.0. A separate 500-skill, roughly 38,000-trajectory evaluation found relevant skills lifting scores by 5.5 to 22 points, with open-weight GLM 5.1 plus a skill scoring 91.1 at about $0.89 per scenario against Opus 4.8 at 92.7 for $3.26.

That last number is the under-discussed lever. If you pair a cheaper model with a strong procedure, you can approach frontier quality at a third to a quarter of the cost — which changes the economics of Autoprompt’s ~2x token multiplier far more than shaving one subagent will.

For context on why this market exists at all: Anthropic’s 2026 survey of more than 500 technical leaders found about 90% of organizations use AI for coding, 86% deploy coding agents for production code, and 57% run agents on multi-stage workflows. Time savings were reported at 59% across code generation, review and testing, and research alike. Integration with existing systems (46%) is the top adoption blocker, ahead of data quality (42%) and cost (43%).

The npm Versus GitHub Distribution Gap

If you install this tool with the command most articles still publish, you get the old version.

npm weekly downloads were 38 for the week of 2026-09-21 to 2026-09-27, against third-party trackers still publishing the older figure of 535 per week — a 14x discrepancy that mostly reflects how slowly catalog data propagates.

Repository state on the date this review was verified, 2026-09-28: 1,295 stars, 88 forks, 3 open issues, MIT license, 44 commits, created 2026-08-17, latest release v2.0.0 on 2026-09-09. Star velocity at launch was cited at 437 stars in three days, roughly 153 per day.

The version divergence explains most of the factual disagreements between reviews of this tool. Published comparisons disagree on provider count (9 versus 11) and star count (590 versus 1,300) because they were written against different points in a fast-moving repository. Some also repeat unattributed anecdotes — debugging-time reductions at an investment firm, e-commerce reliability gains — with no source at all. The 45% figure is exactly the kind of claim that survives this propagation intact while every verifiable detail around it goes stale; the version data in this review is dated deliberately so you can see how far it has drifted since.

Where Autoprompt Earns Its Cost, and Where It Does Not

The category literature says a minority of skills genuinely help and the winners are concrete workflows, not general best-practice documents. Autoprompt is positioned in the subcategory that Anthropic reports as delivering the largest quality improvement: verification-type skills. That is the strongest structural argument in its favor, and it does not depend on the 45% at all.

The strongest practical argument is role separation. A reviewer that runs the code and can disagree on evidence is a different thing from a model approving its own output, and that architectural difference is what makes the benchmark result plausible — plan, implement, and verify as distinct calls with distinct context. Autonomy here is bounded by design: the workflow stops for choices that change the result or actions that require authority. If you were sold “fully autonomous coding,” that is not what this is, and the stopping behavior is a feature.

Where it does not earn its cost:

• Small, well-specified changes. The project’s own caveat is that small tasks “may differ significantly,” and the efficiency-regression literature is dominated by exactly this pattern. Use path=direct or skip it.
• Tasks with nothing to verify. No executable check means the verification phase buys an opinion, not evidence.
• Windows development teams. The verified surface is Linux; three open items say so.
• Workflows needing an auditable number. The per-task evidence is 404 and v2 has no benchmark. You are trusting a methodology document.
• Multi-skill setups. If you already load four or more skills, the incremental value measured by SkillsBench drops sharply, and skill metadata competes for context — Claude Code caps skill metadata at roughly 1% of the context window (~2,000 tokens, about 15-25 skills on a 200K model) before descriptions get truncated.

Verdict: Who Should Use It in 2026

Autoprompt is a well-engineered orchestration skill with a real, single-run accuracy result, unusually candid disclosure about its own gaps, and a version 2 that has not been benchmarked. The 45% is defensible only with its denominator attached: 13 tasks out of 89, on one harness, with one model, costing an unmeasured ~3x time and ~2x tokens.

Adopt it if you run long agentic coding sessions on Linux, you already have test suites the verifier can execute, and you want structural role separation between implementation and review. Start with the default tokensaver concurrency, pin --max-subs to something honest for your change size, and measure your own token delta in week one.

Do not adopt it on the strength of the headline. Do not cite it as an established result without noting that the evidence directory is gone and v2 is untested. And if you need independent confirmation before standardizing, rerun Terminal-Bench on your own repositories — which is what the project’s own disclosure effectively invites you to do.

The honest one-line summary: this is the exception the skeptical literature allows for, priced at 3x the wall clock, sold with a number that is two releases old.

FAQ

Does Autoprompt really reduce coding agent failures by 45%?

On one measured run, yes: failures fell from 29 to 16 across 89 Terminal-Bench 2.1 tasks using OpenCode 1.18.7 and DeepSeek V4 Flash, a 44.8% reduction. That is a single benchmark on a single harness and model combination, and the project itself labels it a version 1 result. The same 13-task improvement is also described as “about 2x fewer mistakes.”

Is the 45% benchmark independently verifiable?

Not from the project’s own artifacts. The per-task verdict file the benchmark docs link to returns HTTP 404, and the benchmark/ directory is absent from the main branch among 1,381 paths in the recursive tree. The methodology document remains, so you can read how the run was done, but the raw verdicts needed to audit it are no longer published. Independent verification means rerunning the benchmark yourself.

How much more expensive is Autoprompt than a plain coding agent?

The disclosed figure is roughly 3x wall-clock time and 2x tokens — and the project states plainly that these are planning estimates from user experience reports, not measured benchmark results, because timing and token logs were not retained. Actual overhead depends mostly on fan-out, which you control via the tokensaver (six subagents maximum), wide, or custom --max-subs N concurrency profiles.

Which coding agents does Autoprompt support, and on which platforms?

v2.0.0 claims 11 providers, extending the original Codex, OpenCode, and Prime Agent list and adding Hermes Agent and Grok Build. Platform verification is narrower: the release notes concede native runtime verification remains Linux-only, and the three open repository items all concern Windows — PROVIDER_UNSUPPORTED activation failures caused by linux-only trusted keys, an OpenCode activation failure on the npm cmd shim, and an in-flight PR fixing Windows npm shims.

Should I install Autoprompt from npm or GitHub?

GitHub. npm still ships 1.0.4, published 2026-08-21, while the repository is at v2.0.0 from 2026-09-09 — a two-major-version gap that means the npm package lacks the CLI, the path controls, and the expanded provider support. npm weekly downloads were 38 for the week ending 2026-09-27, so the stale channel is also the less-used one.