Enterprise AI coding ROI must be measured for the work and costs actually affected. A tool can accelerate part of development without producing an equivalent financial return across the organization. This guide separates research findings, planning assumptions and realized value.
Correction — October 7, 2026: The earlier article attributed a 376% ROI figure to GitHub Copilot alone, asserted universal enterprise ROI ranges and treated shorter PR cycle time as directly billable labor savings. Those statements were not supported as written. The research scope and calculation method below replace them.
What the 376% Forrester Result Actually Covers
The July 2025 Forrester Total Economic Impact study was commissioned by GitHub and assessed GitHub Enterprise Cloud. It reports 376% ROI over three years for a modeled composite organization built from interviews with five decision-makers at four organizations.
That scope includes an integrated platform and multiple benefits, not a randomized estimate of Copilot’s isolated effect. It is a useful example of a business-case model, not a return that every coding assistant or enterprise should expect. GitHub’s summary also names Enterprise Cloud as the product being assessed.
Do not convert this result into “most teams earn 3–6x,” a guaranteed payback period, or a market-wide success rate. The earlier average/top-quartile tables and “only 5% succeed” statement have been removed because their definitions and comparable primary evidence were insufficient.
Productivity Research Is Specific to Its Setting
METR’s July 2025 randomized study involved 16 experienced open-source developers and 246 tasks in repositories they knew. Allowing early-2025 AI tools increased completion time by 19% in that setting. The authors explicitly caution against generalizing the result to all developers, tasks or later tools.
There is also a February 2026 follow-up. That update does publish later estimates: an 18% speedup for the original developers who took part again (confidence interval −38% to +9%) and a 4% speedup for newly recruited developers (confidence interval −15% to +9%). METR itself calls this only very weak evidence and says selection effects, a reduced pay rate and unreliable time measurement make the central estimate likely a bad proxy for the real productivity impact. The original slowdown is therefore a historical result with a defined scope, not a fixed penalty to apply to every 2026 business case — and neither is the later speedup a figure to import.
For a team evaluation, distinguish what people feel from what you measure, and retain both. Reported satisfaction can be valuable even when delivery speed does not change. Conversely, faster generation may be offset by additional review or rework. Test those possibilities instead of assuming which explains your result.
Define the Outcome Before Measuring the Tool
Choose a problem that the proposed use of AI could actually affect. Examples include reducing active time spent on routine test scaffolding, improving the success rate of a migration task, or reducing support escalation caused by an internal tool. Define acceptable quality and the time period in advance.
Then distinguish three kinds of evidence:
| Evidence | What it can establish | What it does not establish alone |
|---|---|---|
| Controlled task evaluation | Performance on the selected tasks and setup | An organization-wide cash return |
| Observed delivery metrics | Changes in the production workflow | Causation when staffing, demand or process also changed |
| Financial records | Actual costs, avoided spend or attributable revenue | Which feature caused a change without supporting analysis |
Count task failures, human review and correction. A successful run’s time is not a representative average if failed attempts are omitted.
Delivery Metrics and Developer Experience
DORA’s current guide uses five software-delivery metrics: change lead time, deployment frequency, failed deployment recovery time, change fail rate and deployment rework rate. Use them at the application or service level and preserve their definitions when comparing periods. They describe delivery performance; they do not automatically calculate financial ROI.
Add direct measures of the work affected: active implementation time, review time, rework, escaped defects and developer satisfaction. Raw commit counts or generated lines of code can rise without better outcomes, so use them as supporting context rather than a financial benefit.
PR cycle time often includes waiting. A decrease in elapsed cycle time cannot simply be multiplied by salaries and called labor savings. Identify which active work hours were reduced, whether the released capacity was used, and whether any expenditure or revenue actually changed.
A Cost Ledger That Captures the Whole Workflow
Track costs over the same period as benefits. Suggested categories include:
- Seats, metered model usage and paid agent services.
- Integration, training and workflow changes.
- Human review, correction and failed attempts.
- Infrastructure, observability and maintenance.
- Incremental security or support work that can be attributed to the rollout.
These are categories to investigate, not claims that every deployment incurs the same cost or vulnerability rate. The earlier fixed figures for PR delays, security findings and enterprise failure rates have been removed. For the per-seat and per-token side of this ledger, see the per-developer AI coding cost breakdown, which itemises subscription, metered usage and infrastructure lines without asserting a return.
Separate incremental spending from existing overhead. Also separate a one-time setup cost from ongoing costs. If both are included in an evaluation, state how they are allocated over the period rather than silently amortizing them in whichever way improves the headline return.
Calculating Value Without Double Counting
For an explicitly defined period, a simple calculation is:
ROI = (attributable benefits - incremental costs) / incremental costs × 100%
For multi-year cash flows, use a consistent discounting approach and state the assumptions.
Suppose a pilot with 20 developers measures two fewer active work hours per developer per week across four weeks. At a hypothetical loaded rate of $60/hour, the gross capacity value is 20 × 2 × 4 × $60 = $9,600.
If you assume only 50% of that capacity becomes useful additional work, the modeled benefit is $4,800. With $3,000 of incremental costs, the modeled ROI is 60%: ($4,800 - $3,000) / $3,000.
| Assumed realization of measured capacity | Modeled benefit | Modeled ROI with $3,000 cost |
|---|---|---|
| 25% | $2,400 | -20% |
| 50% | $4,800 | 60% |
| 75% | $7,200 | 140% |
The realization percentages are assumptions for sensitivity analysis. They are not published enterprise benchmarks. This modeled capacity value is not cash saved unless expenditure was actually avoided or reduced. Do not also count the same hours as separate delivery and incident benefits.
A runnable capacity-value model
The arithmetic above is trivial; the disagreement in a review is almost always about the inputs. To make those inputs inspectable instead of buried in a private spreadsheet, this guide ships a small deterministic model you can run on your own headcount, loaded rate, measured capacity and cost lines. It is a stdlib-only single file with no network access and no dependency installs.
| File | What it is |
|---|---|
| roi-model.py | The model: capacity in, modeled annual benefit and ROI out, CSV only |
| inputs.example.csv | Synthetic example inputs, one row per scenario, units documented |
| assumptions.json | Every inference, each flagged is_assumption: true |
| expected-output.csv | Published output, byte-for-byte reproduced below |
| README.md | Commands, interpreter version, output hash, limits |
python3 roi-model.py --inputs inputs.example.csv \
--assumptions assumptions.json --out expected-output.csv
Verified on 2026-10-25 with CPython 3.12.3 on Linux x86_64, offline, repeated twice: both runs produced expected-output.csv (458 bytes) with sha256 b406e56d7e55d4ae6270a8794e23f0cb62991f1676b5122c95e2f2b608117819. That is reproducibility of a model, not a measured enterprise result — the run shows the arithmetic is deterministic and the inputs are visible, nothing more.
The example rows, reproduced from that output, use 20 developers, a $60 loaded hourly rate, 2.0 measured capacity hours per developer per week and $25,800 of incremental annual cost:
| Scenario | Realization | Modeled benefit / year | Modeled ROI | Modeled payback | Benefit is cash? |
|---|---|---|---|---|---|
| low-realization | 25% | $20,700 | -19.77% | 14.96 months | no |
| base | 50% | $41,400 | 60.47% | 7.48 months | no |
| high-realization | 75% | $62,100 | 140.70% | 4.99 months | no |
| review-heavy | 50%, 1.5h review overhead/dev/week | $13,800 | -46.51% | 22.43 months | no |
The review-heavy row exists to keep one failure mode visible: the first three rows subtract 0.5 review hours per developer per week from the 2.0 measured capacity hours, and the fourth subtracts 1.5. That single change cuts net capacity from 1,380 to 460 hours a year and moves the same realization dial from +60.47% to -46.51%. Pushing review overhead past the measured capacity is not modelled at all: the script exits instead of reporting negative capacity as a benefit. If a model can only show a positive result, it is not a sensitivity analysis.
What the model does not do
| It does | It does not |
|---|---|
| Annualise a capacity number you measured | Measure productivity or estimate capacity for you |
| Subtract added review overhead from gross capacity, once | Detect selection effects or task-mix drift in your measurement |
| Apply a realization dial you set and can defend | Supply a benchmark realization rate; no vendor figure is a default |
Report benefit_is_cash: no on every row | Prove avoided expenditure or attributable revenue |
| Annualise at a stated 46 working weeks | Model cash-flow timing or produce NPV and IRR; a multi-year case needs an explicit discount rate outside the file |
The model also rejects any row where review overhead exceeds measured capacity, because net negative capacity is not a benefit. Treat the saved capacity as the first question and the savings side as a separate one: reducing token and seat spend is its own exercise, set out in AI developer cost optimization, and it should not be added to the same hours.
Comparing the Studies You Will Be Quoted
Three sources dominate this conversation and they answer three different questions. Putting them in one table is the fastest way to stop a modeled vendor figure, a measured slowdown and a set of delivery metrics from being cited as if they were the same kind of evidence.
| Source | Population | Design | Metric | What it cannot establish |
|---|---|---|---|---|
| Forrester TEI for GitHub Enterprise Cloud (July 2025, commissioned by GitHub) | Composite organisation of 5,000 developers assembled for the study from interviews with four organisations | Commissioned Total Economic Impact financial model, three-year horizon, assumed adoption and productivity-recapture inputs | ROI and NPV in dollars; payback in months for the integrated platform | A causal effect of the coding assistant alone, or what your organisation will earn; the sponsor chose the study and the composite |
| METR RCT, July 2025 | 16 experienced open-source maintainers working in their own well-known repositories | Randomised controlled trial, 246 tasks, early-2025 chat, agent and autocomplete tools | Task completion time: 19% longer with AI | Anything about enterprise codebases, junior developers, greenfield work or later tool generations, and any financial return — the authors warn against generalising |
| METR design update, February 2026 | Second study from August 2025: 10 developers from the original cohort plus 47 newly recruited from a more diverse set of open-source projects, paid $50/hour | Same pre-specified task randomisation as the first study, with the authors’ own account of a design they now consider unreliable | Self-reported task completion time: an 18% speedup for returning developers (CI −38% to +9%) and a 4% speedup for newly recruited ones (CI −15% to +9%), which the authors call very weak evidence | A reliable current effect size: the authors say selection effects, a lower pay rate and unreliable time measurement make the central estimate likely a bad proxy for real productivity impact |
| DORA delivery metrics | Delivery teams measuring at application or service level | Metric definitions for delivery performance, not a tool study | Change lead time, deployment frequency, failed deployment recovery time, change fail rate, deployment rework rate | Any monetary value, and any causal attribution to an AI tool when staffing, demand or process also moved |
Two readings follow. A vendor-commissioned TEI model and a vendor-neutral RCT can both be honest and still disagree, because one models a platform’s value with assumed inputs and the other times tasks under one tool era. And DORA’s five metrics are the right instrument for “did delivery change”, which is why they belong next to the cost ledger rather than inside a dollar figure: none of them is a monetary measure, so the mapping from delivery change to cost or capacity has to be stated separately.
Running a Credible Pilot
Record the baseline, eligible tasks, participating teams and tool versions. Where practical, compare similar work with and without the change. Keep quality criteria and review requirements stable, and log material changes to the workflow or task mix.
A review should answer four questions: Did acceptable work become easier or faster? Did quality or reliability change? What did the intervention cost? Which benefits were realized, and which remain modeled assumptions?
Report the range of outcomes and the uncertainty, not just an average across incompatible tasks. If the sample is small or changes are confounded, extend or redesign the evaluation. Avoid assigning a universal ROI expectation based only on team size or how long a subscription has been active.
Building the Leadership Business Case
Present the current problem, the proposed intervention, the evaluation design and the decision criteria. Include a conservative scenario and the variables that could make the result worse. Name who owns the cost ledger and who verifies claimed benefits.
Scale a use case when the evidence supports it and the relevant constraints are understood. If the pilot only demonstrates useful capacity rather than cash savings, describe that outcome directly. A transparent limited result is more useful than a large percentage assembled from unrelated studies.
FAQ
Does Forrester prove that Copilot alone delivers 376% ROI?
No. The cited commissioned study assesses Enterprise Cloud and a modeled composite organization. Its platform-level result does not isolate Copilot’s causal contribution.
Is time saved the same as money saved?
No. Released capacity can be valuable without reducing cash expenditure. Record the actual use of that capacity and avoid treating it as realized cash automatically.
Does METR show that current AI tools always slow developers down?
No. Its early-2025 result applies to the studied developers and tasks. The follow-up also describes important limits in estimating later effects.
What is a realistic ROI target?
Set a target from the affected workflow, measured baseline and explicit cost/benefit assumptions. This article does not supply a defensible universal return by team size.
Does the runnable model prove an ROI?
No. It reproduces a calculation from inputs you supply and reports benefit_is_cash: no on every row. Its example inputs are synthetic, so its output illustrates mechanics; it is not a measurement of any organisation’s return and it should not be quoted as one.