Tue, 07 Jul 2026 21:00:00 +0000

If you’re running Copilot Enterprise across a few hundred engineers, you’ve probably stared at the usage dashboard and asked yourself: can I trust these numbers? The short answer is yes, with caveats. The longer answer is what this article is about.

I’ve been working with Copilot’s metrics pipeline across several mid-to-large enterprise deployments, and I keep running into the same three questions: How accurate are these metrics? What’s actually covered by the data? And how do I build reliable team-level reports when the API doesn’t give me a team endpoint? This guide tackles each of those head-on.

Accuracy: What the Numbers Actually Mean

The first thing to understand is that Copilot usage metrics come from two separate telemetry pipelines, and they don’t tell the same story.

Client-Side vs Server-Side Telemetry

Client-side IDE telemetry is the primary source. When a developer uses Copilot in VS Code, JetBrains, or another supported IDE, the extension sends events directly to GitHub — completions accepted or rejected, chat requests sent, agent mode sessions started. This is where the rich data lives: language breakdowns, model usage, LoC metrics, the works.

Server-side telemetry is the fallback. GitHub detects activity through server-side signals — API calls, license checks, Copilot Chat on GitHub.com. It confirms a user is active, but that’s about it.

Here’s the problem: server-side users count fully toward your Daily Active Users (DAU) and Weekly Active Users (WAU), but their dimensional breakdowns are empty. You’ll see a user in your active count with zero lines of code, zero chat requests, zero completions attributed to them.

In practice, I’ve seen 5-15% of DAU come from server-side only in standard setups, and up to 20% in heavily locked-down enterprises with restrictive proxies or custom network policies. If you’re running a weekly report that shows average chat requests per active user, that average is artificially low because the server-side cohort drags it down with zeroes.

What to do about it: When computing per-user engagement metrics, I filter to users who have at least one client-side telemetry event in the period. The denominator becomes “users with measurable activity” rather than “all active users.” This gives a more honest picture of engagement depth, even though it lowers your headline adoption numbers slightly.

The 2-Day Data Lag

Data is available within 2 full UTC days after the day closes. Monday’s numbers land on Wednesday morning. This isn’t a bug — it’s the processing pipeline doing deduplication, attribution, and aggregation.

The practical impact: don’t build real-time dashboards. I’ve seen teams try to wire Copilot metrics into a live Grafana panel and then panic when Tuesday afternoon shows zero data for Monday. Build daily or weekly batch reports instead. A Monday-morning script that pulls the previous week’s data (Saturday through Friday, accounting for the 2-day lag) gives you clean, consistent snapshots.

Attribution Accuracy: The Org Membership Trap

This is the one that trips up most engineering leaders. Organization-level metrics are based on organization membership, not on where the user’s actions occurred.

Concrete example: Alice belongs to the “Platform Engineering” org and the “Security” org. She writes code in a Platform Engineering repo. Her usage appears in both org dashboards. If you sum the DAU across all your orgs, you’ll overcount Alice twice.

The enterprise-level endpoint deduplicates users. The org-level endpoints don’t. This is by design — org-level metrics are meant for visibility into adoption within a specific org, not for enterprise-wide rollups. But if you’re building a consolidated report by summing org dashboards, your numbers will be wrong.

The fix: Always use the enterprise-level endpoints (/enterprises/{enterprise}/copilot/metrics/reports/...) for your top-line numbers. Use org-level endpoints only when you need to understand adoption patterns within a specific org, and never sum them.

Coverage: What Gets Counted and What Doesn’t

Coverage is the second dimension of accuracy. Even when the telemetry is working correctly, it doesn’t cover every scenario.

IDE and Version Coverage

LoC metrics require minimum IDE versions. If your team hasn’t upgraded, you’re flying blind on lines-of-code data:

IDE	Minimum Version
VS Code	1.104.0
JetBrains	2024.2.6
Visual Studio	17.14
Xcode	14.3.1

In one deployment I worked with, about 30% of the JetBrains users were on 2024.1.x because the team had a policy of staying one major version behind. Those users generated completions and chat activity, but their LoC metrics were completely absent. The dashboard showed healthy adoption numbers but suspiciously low LoC totals.

What to check: Run a quick IDE version survey across your org. If more than 10% of your users are below the minimum versions for LoC metrics, factor that into your reporting. Your LoC numbers are an undercount, and the gap will shrink as users upgrade.

Agent Mode Coverage

Agent mode doesn’t follow the “suggest then accept” flow that inline completions use. Instead of a suggestion event followed by an accept or reject event, agent mode edits are counted as loc_added_sum and loc_deleted_sum under the agent_edit feature bucket.

This means agent mode activity won’t show up in your acceptance rate metrics at all. If your team is heavy on agent mode (and more teams are every quarter, based on what I’m seeing), your acceptance rate will look lower than the actual value Copilot is delivering. The code is being written — it’s just not being measured the same way.

For more on how agent mode changes the Copilot experience, see the Copilot Agent Mode guide.

PR Lifecycle Coverage

Pull request lifecycle metrics come from a completely different pipeline than IDE telemetry. PR data is derived from repository activity on GitHub.com. This means a developer who uses Copilot CLI to write code, commits via the terminal, and opens a PR through the web UI will show Copilot attribution on the PR even if their IDE never reported a single telemetry event.

This is actually a feature, not a bug — it captures Copilot usage that IDE telemetry would miss. But it creates a confusing reporting situation where your PR metrics show more Copilot activity than your IDE metrics do. If you’re reconciling the two, expect PR-based metrics to be 5-15% higher than IDE-based metrics in orgs with significant terminal-based workflows.

Team-Level Reporting: Building It Yourself

There is no dedicated team-level endpoint in the Copilot Usage Metrics API. If you want per-team numbers, you build them yourself by joining two reports.

The Join Approach

The method is straightforward:

Download the user usage report for a given day (per-user metrics)
Download the user-teams report for the same day (which teams each user belongs to)
Inner-join on user_id, then group by team_id

The user-teams endpoints are:

Here’s a production script I use for weekly team reports:

import pandas as pd
import json
from datetime import datetime, timedelta

def build_team_report(org: str, date: str, token: str):
    """Build per-team Copilot metrics for a given day."""
    # Download reports (omitting curl logic for brevity)
    users_df = pd.read_json(f"users-{date}.ndjson", lines=True)
    teams_df = pd.read_json(f"user-teams-{date}.ndjson", lines=True)

    # Explode team memberships — one row per user-team pair
    team_members = teams_df.explode("teams")

    # Inner join: only users with both metrics and team membership
    joined = team_members.merge(
        users_df,
        left_on="user_id",
        right_on="id",
        how="inner"
    )

    # Aggregate by team
    team_agg = joined.groupby("teams").agg(
        active_users=("user_id", "nunique"),
        avg_acceptance_rate=(
            "copilot_ide_code_completions_acceptance_rate", "mean"
        ),
        total_chat_requests=(
            "copilot_ide_chat_total_requests", "sum"
        ),
        total_completions=(
            "copilot_ide_code_completions_total_engaged_users", "sum"
        )
    ).reset_index()

    return team_agg

Critical Gotchas

Teams with fewer than 5 seated Copilot users are excluded from the user-teams report. If you have a small team (a 3-person SRE squad, for example), they simply won’t appear. This is a privacy safeguard from GitHub, and there’s no way around it.

Users on multiple teams contribute to each team’s totals. If Alice is on both “Frontend” and “API” teams, her usage counts in both. Summing team rows will double-count her. When you present team-level data, make sure your audience understands this — or use COUNT(DISTINCT user_id) for distinct-user metrics and accept that team-level distinct counts won’t sum to org-level totals.

Team membership changes over time. If you’re building rolling 28-day team reports, join each day’s reports separately rather than joining once and assuming membership is stable. A user who moved teams mid-month will have their early-month activity attributed to the old team and late-month activity to the new team if you join per-day. If you join once and apply the same membership to all days, you’ll misattribute.

For a broader look at setting up Copilot across your organization, see the GitHub Copilot Enterprise Guide.

Practical Recommendations

After working through these accuracy, coverage, and reporting challenges across several orgs, here’s what I’d recommend:

1. Document your data sources. In every report you produce, note whether the numbers come from client-side telemetry, server-side telemetry, or a mix. Your stakeholders need to know that “active users” and “users with LoC data” are different populations.

2. Run a weekly reconciliation. Every Monday, pull the enterprise-level 28-day report and compare it against the sum of your org-level reports. The gap between the two is your cross-org deduplication rate. If it’s growing, your users are joining more orgs — which might be worth investigating.

3. Track agent adoption separately. Since agent mode doesn’t contribute to acceptance rate, create a separate dashboard for agent activity. Watch the ratio of agent edits to inline completions over time. A rising ratio means your team is moving toward autonomous AI workflows, which changes how you should interpret the traditional metrics.

For more on how agentic workflows change the metrics picture, check out the Agentic Code Review guide.

4. Accept the 2-day lag. Build your reporting cadence around it. A weekly report that covers Monday through Sunday, pulled on Tuesday, gives you complete data for the previous week. Don’t fight the pipeline — work with it.

5. Cross-reference with DORA metrics. The real ROI story isn’t in Copilot’s dashboard — it’s in your deployment frequency, lead time, and change failure rate. If you’re already tracking DORA metrics, overlay your Copilot adoption data. Teams with high Copilot engagement that also ship faster with stable failure rates are your success stories. The general usage metrics guide covers the five metric categories in more detail if you need a refresher.

The Bottom Line

GitHub Copilot’s metrics are the most mature of any AI coding tool on the market, but they’re not a turnkey solution. The dual telemetry pipelines, the org attribution model, the IDE version requirements, and the lack of a native team endpoint all mean you need to understand the data before you can trust it.

Build your reporting around the enterprise-level endpoints, account for the 2-day lag, filter server-side users when computing per-user engagement, and build team-level reports through the user-teams join. Do that, and you’ll have a solid, defensible view of whether Copilot is actually making your engineering organization faster.

Data Accuracy on RockB