Unreal Agent Runtime Review 2026: An Async-First Harness You Deploy, Not Drive

Unreal Agent is an async-first agent harness from Unreal Labs: a Go library, a runner binary, and a Harbor-compatible benchmark runner, MIT licensed, first released on 22 September 2026. When the model issues a tool call, the harness immediately appends an event-log record in the in-progress state, lets the model keep working, and appends the real result when the tool finishes. It is not Epic’s Unreal Engine. It is also not Inworld’s Unreal Engine AI toolkit, which Inworld launched as the “Unreal AI Runtime” — two unrelated products already occupy that name.

This article is a source-verification review, not a hands-on trial. The harness was not installed and no benchmark was re-run for it. Every figure below was re-read from the vendor’s own published artifacts on 12 October 2026, and the pass/fail arithmetic was recomputed from the vendor’s published totals rather than quoted from a headline. The re-checks were a private local author verification pass, not a published dataset: no downloadable log, fixture or bundle ships with this article or is linked as a reader resource. The exact URLs and their read times are listed in the sources section at the end so that any reader can open the same pages and reproduce each comparison. The only claim this article makes about behaviour is the claim the vendor and its issue tracker make.

What Unreal Agent Is, and What It Is Not

A harness is everything around the model: the prompt, the tool loop, the context assembly, the session state, the sandbox — the AI harness engineering guide covers that boundary in more detail. The model is the same in every harness; the harness decides how many turns a task takes, how many input tokens each turn re-sends, and what survives a crash.

The project ships three things, per its own repository and README:

• A Go library (harness/) that you embed in your own service. The module path is github.com/unreallabsai/unreal-agent and go.mod declares go 1.27.0 with four direct requires (oapi-codegen/runtime v1.6.0, go.yaml.in/yaml/v3 v3.0.5, golang.org/x/image v0.46.0, golang.org/x/sys v0.48.0) — a small dependency tree for a runtime, which the vendor lists as a design goal.
• A runner executable (unreal-agent-runner), plus a TUI (unreal-agent-tui). It writes event JSONL to stdout and exits when the task finishes, which makes it scriptable with jq in the same way as codex exec or claude -p.
• A Harbor benchmark adapter under benchmarks/harbor, so the same harness can be scored on the suites Harbor hosts.

What it is not: it is not a hosted product with a sandbox and an IDE extension, and it is not a model. The provider is selected at runtime — the runner README documents UNREAL_HARNESS_LLM_PROVIDER as openai, openai-codex, openrouter, fireworks, or ollama, with the model chosen by UNREAL_HARNESS_LLM_MODEL.

Repository velocity, read from the GitHub repository API on 12 October 2026: 2,190 stars, 127 forks, 8 open issues, language Go, MIT licence, last push 2026-10-06T17:28:03Z, repository created 2026-09-21T23:14:02Z. The same counters read on 1 October 2026 were 2,049 stars, 118 forks and 8 open issues, so the project gained 141 stars and nine forks in eleven days while its open-issue count stayed at eight — new bugs arrived at roughly the rate the old ones were closed. The commits API reports 189 commits on main at this read, against roughly 135 on 1 October. The contributors API returns six entries as of this read: g-logunov, maxim092001, sashabaranov, artyom-ivanov, discobot and dependabot[bot]. That is a small team, and it is worth saying plainly rather than describing the project as a community.

The Cost Claim, Recomputed From the Vendor’s Own Tables

The launch post claims the harness “delivers up to 40% cost savings compared to Codex on production workloads and coding/science benchmarks, without any negative performance impact”, and “up to 20% compared to Pi”. The post then publishes four per-benchmark tables. All four were run on GPT-6 Astra with xhigh reasoning effort, and three of them carry Harbor job IDs that a reader can look up.

Terminal-Bench 4.0, vendor-run through Harbor:

At an identical published pass rate of 57.9%, that is a straightforward recomputation: 1 − 1428/2350 = 39.23% cheaper than the Codex leaderboard row, and 1 − 1428/1827 = 21.84% cheaper than the vendor’s own Pi run. Per benchmark point, 1,428 ÷ 57.9 is $24.66 for Unreal Agent against 2,350 ÷ 57.9, or $40.59, for Codex.

The same arithmetic across the other three suites, using the totals published in the launch post:

The direction is consistent, which is the part that makes the claim interesting: the harness does not win on one lucky suite. But the spread is the story, not the 40 in the headline. Against Codex the four rows run 39.2%, 28.2%, 16.3% and 25.7%; against the vendor’s own Pi configuration they collapse to 21.8%, 9.4%, 13.7% and 17.2%. A “40% cheaper” sentence is true only of the Terminal-Bench row and only against the Codex leaderboard baseline, which has no published job ID and no token columns at all — you cannot inspect how that number was produced, only compare against it.

Where the Headline Is Weaker Than It Looks

Four qualifications belong on those tables, and the vendor supplies most of them itself.

The runs are self-run. Harbor is a real comparison framework and the job IDs are published, but the party being measured chose the configuration, the benchmark version and the model. The launch post is candid about the pass-rate differences — “marginal differences in pass rate, which we attribute to benchmark variance” — which is honest but also means the cost comparison rests on roughly equal scores rather than on matched runs.

One model, one effort level. GPT-6 Astra at xhigh throughout. Nothing is published about how the harness behaves on a cheaper or weaker model, which is exactly where harness efficiency matters most per dollar. There is a second trap here for anyone attempting a re-run: the Harbor adapter README documents thinking_level as low, medium, high (default), xhigh, or max, while the published tables were run at xhigh — and the README’s own Terminal-Bench on Modal example passes thinking_level=max. Copy the README’s example verbatim and you have changed the controlled variable twice.

The fourth row is not on Harbor. The ALE-CLI table states it directly: “These runs are not on Harbor.” No job IDs, the smallest sample, and therefore directional evidence only. It should not carry the cost argument.

The headline chart shipped wrong. Within hours of the launch, a commenter on the Hacker News thread asked whether the comparison was effort-matched, writing that it looked like the vendor was comparing its harness on Astra xhigh against Codex on Astra max. An Unreal Labs account acknowledged it in the same thread: “the headline chart was lazy on our part, thanks for the flag and I’ll update it.” The per-benchmark tables were run at matched xhigh; the summary chart was not. The right response from a reader is to ignore the chart and read the rows — and to treat any vendor summary number as unverified until the row underneath it can be found.

Why Async Dispatch Could Save Tokens — and What the Vendor Actually Claims

Traditional harnesses are synchronous: the model emits a tool call, the harness runs it, and only when it finishes does the loop issue the next model turn. Unreal Agent inverts the order. The vendor’s own description: every tool call “immediately append[s] an event-log record that the tool has returned in the ‘in-progress’ state, while continuing its execution in the background”, with the real result appended when the tool finishes. The post adds a line that is more revealing than anything in the benchmark tables: “Making this work without breaking cache was an interesting engineering challenge in itself.”

That cache line is the real mechanism, and it is worth separating from the marketing version of it, because a sharp commenter in the launch thread did not accept the simple story. Asked to explain “how async tool calls translate to token savings”, the skeptical reading is that a synchronous harness is not burning any tokens while it waits: the turn is already complete. That is correct. The saving is not the wait. It is that a harness which blocks on a long tool call and then re-sends a large conversation can fall out of the provider’s prompt cache, while a harness that never blocks keeps the growing context inside the warm window. The vendor frames its whole cost argument this way — “fewer model turns and fewer input tokens” — and the token columns are where that shows up.

Recomputing the input-token rows from the published tables gives a tighter range than the vendor’s own headline — “up to 40% cost savings compared to Codex and up to 20% compared to Pi” — because that headline is a claim about total spend across all four suites, not about input tokens on any one of them (the mechanics of which turns get re-charged are set out in the agent token cost attribution guide):

• Terminal-Bench 4.0: 1.73M against Pi’s 2.83M is 38.9% fewer input tokens per trial. Codex has no token column on this table, so no Codex token comparison exists there.
• SWE-Atlas Codebase QnA: 898k against Codex’s 1.69M is 46.9% fewer, and against Pi’s 1.29M it is 30.4% fewer.
• DeepSWE 1.1: 1.60M against Codex’s 2.19M is 26.9% fewer, and against Pi’s 2.21M it is 27.6% fewer.
• ALE-CLI: 0.76M against Codex’s 1.59M is 52.2% fewer, and against Pi’s 1.19M it is 36.1% fewer.

So the honest statement is: 26.9% to 46.9% fewer input tokens per trial across the three Harbor rows, with the ALE-CLI row at 52.2%, and 27.6% to 38.9% against the vendor’s own Pi configuration. Turns follow the same curve on Terminal-Bench — 28 against 44, or 36% fewer — and tool calls 37 against 57, or 35% fewer.

What Is Actually Inside: Eight Components and a Deployable Unit Called an Operation

The README names exactly eight replaceable components. That list is more informative than any benchmark table, because it is what you would be adopting.

The load-bearing word is operation: “a serializable description of work produced by a tool translator for asynchronous execution”. An operation is not a function call — it is a document. You could write it to a queue, ship it across a process boundary, or replay it after a crash, and the README states that this is the intent: “a proxy operations manager can send serialized operations to a local operations manager running in a process inside a remote sandbox, allowing tools to execute there.”

The versioning invariants are the part nobody normally ships in month one, and they are stated explicitly: session-store items are serializable with a versioned storage format; operations are versioned and always serializable; and “an unsupported session version will always cause an explicit error on resume” rather than silently mis-parsing. Combined with append-only storage, explicit forking and recovery, that is the property set you need if an agent turn is infrastructure rather than a chat session. The vendor does publish tests for exactly those paths: TestCompactionCrashContinuation in harness/coordinator/compaction_scheduling_test.go reconstructs the session after a simulated crash, TestRunResumesAfterOutputFailure and TestRunMainResumesInterruptedDeliveryWithDuplicateInput in cmd/internal/agentrunner/resume_test.go cover resume after output failure and duplicate delivery, and TestCoordinatorStopCancelsShellProcess with the surrounding cancellation tests in harness/coordinator/stop_integration_test.go and stop_test.go cover the kill path. The repository’s Makefile defines test as go test -race ./..., and its ci.yml runs make test check build on every push to main and on pull requests targeting it. What is not published is a durability measurement: no crash-recovery benchmark, and no kill-path or disk-growth outcome numbers. So the durability story is verified by tests and a versioned store, not by published results. Two of the open issues below sit exactly there.

The System Prompt Is Doing Part of the Work

The harness ships a system preamble at harness/contextbuilder/prompts/preamble.md (raw file), and two of its lines are load-bearing for the cost claim:

• “Each turn re-sends the whole conversation, so prefer to go wider with tool calls — they are cheap — rather than chaining them across a longer sequence of turns.”
• “Tool calls are asynchronous: each starts the moment you issue it and runs in the background, so issuing one never blocks you and many run at once.”

There is also a background heartbeat: “if calls are active and nothing has happened for ten minutes, a heartbeat wakes you”. The vendor’s own summary of the cost mechanism names this directly — “simple prompts, token-optimized tool results, and no sub-agents or workflows” and “a straightforward asynchronous tool-calling model that is clearly explained to an LLM”.

Read those two halves together and the honest conclusion is that the published saving is a bundle: an async runtime plus a prompt that tells the model to batch tool calls and not to poll. No ablated run exists that isolates them — the closest work in this corpus, the Coding Agent Harness Design and Performance ablation study, measures harness components on SWE-Bench Verified rather than on this product — so the split between “the runtime is cheaper” and “the prompt is cheaper” is unmeasured. That is not a debunk — it is the experiment anyone adopting this should run first, because the prompt half is free to copy into any harness.

What Changed Between 1 October and 12 October 2026

This review supersedes an earlier draft written on 1 October 2026, and the project moved in eleven days. The deltas, all read on 12 October:

Two issue-state corrections matter for anyone who read the first draft. Issue #16, “SIGTERM bypasses runner cancellation”, and issue #15, on skill metadata parsing, were both closed on 2026-10-06; the earlier draft listed them as open risks. In their place the tracker gained #22 (a feature request for MCP client support over stdio and HTTP with OAuth, opened 2026-10-07) and #23 (a community example of a sandboxed pull-request review workflow in GitHub Actions, opened 2026-10-12). The Homebrew tap is new, and it arrives with the release automation that shipped on 6 October.

The packaging surface at v0.3.x is unchanged in shape: the runner writes JSONL events to stdout, sessions live under ${XDG_STATE_HOME:-$HOME/.local/state}/unreal-agent/sessions and are overridable with -session-directory, and the Dockerfile is a multi-stage build from a digest-pinned golang:1.27.1-trixie to debian:trixie-slim, with tini as PID 1 and USER 10001:10001. The runner README’s Docker example publishes the image as unrea1labs/unreal-agent:latest — the organisation is unreallabsai, so that image string swaps the l for a 1 and drops the trailing ai — in the vendor’s own README. Small, but it is the kind of detail a v0.3 product ships with.

Production Risk Inventory, Rechecked Against the Tracker

This is the section that matters most if you are considering adoption, because every one of these items is open in the vendor’s own tracker as of 12 October 2026 and several sit on the async dispatch path the product is sold on. The full list was read from the issues API.

The pattern is not “a v0.x project has bugs” — every one does. It is that async execution amplifies them. #3 is a runaway feedback loop inside a durable background process. #11 is a race between two branches of the same parallel fan-out, and it poisons the session permanently rather than failing one call. #13 is a silent failure that only matters because the runner is meant to be driven by automation. A harness that blocks on one tool at a time cannot easily reach those states. The async design is simultaneously the reason to adopt this project and the reason to cap output size, set a disk quota and define a kill path before running it unattended in CI.

How Hard Is the Async Idea Now That OpenAI Documents It?

OpenAI’s Responses API now supports async tool calling: set async: true on a function or custom tool definition and the model can continue working while your application runs the tool, with the result delivered later through a function_call_output item carrying the original call_id. The primary documentation draws the boundary explicitly: “Your application still executes the tool. Async tools don’t move execution to OpenAI or manage your background jobs.”

Two consequences follow, and they point in opposite directions.

It weakens the novelty claim. If the model provider will let a tool call run in the background, a harness whose headline feature is asynchronous dispatch is partly replicating a platform primitive — and commenters in the launch thread made exactly this point, noting that other harnesses have had async tools for a while and that in practice “the agent will often simply call wait() immediately after, and also do it with a longer timeout”, which turns the async call back into a blocking one with extra failure modes.

It also explains why the architecture, not the async trick, is the durable claim. The API boundary stops at “your application still runs the job”: the process lifecycle, the sandbox, the recovery, the queue and the durable record stay with you. That is the gap a runtime fills and the gap a model provider will not close, because closing it means running arbitrary code. And the surface is genuinely underspecified today: the vendor’s own footnote records that “the use of two tool-call result items (one in progress, one final) in a single context is underspecified in the Responses API documentation”, and that during testing they “encountered rejections with some models on some inference providers (not OpenAI)”. Anyone building cross-provider async dispatch is building on a partially documented contract, and the workaround — two result items for one call — is exactly what issue #11 shows failing under load.

What It Does Not Solve

Three deliberate omissions in the design are worth stating as limitations rather than as bugs.

No sub-agents and no workflow layer. The vendor’s argument is that sub-agents multiply context; the payoff shows up in the turn counts. The counter-argument is the open one: a reviewer in the launch thread wanted to see behaviour “on long-horizon tasks where the ’no sub-agents’ constraint starts to bite”. Delegation exists because a single context loses the thread on tasks that outrun the window. A harness that refuses delegation deserves measurement on tasks longer than a benchmark trial, and none is published.

The workspace is not a sandbox and approvals are out of scope by design. The vendor’s stated position is that deterministic constraints — allowed hosts, granular tokens, approval-gating proxies — beat hook-based approvals, which is defensible and also means the constraints are yours to configure around the tool. Issue #5 is the reminder that loading a repository’s own environment file into a process that can run tools is a trust decision, not a supported feature.

There is no outside contribution path. CONTRIBUTING.md states that the repository “contains selected components from a larger internal codebase” and that “we don’t currently have the capacity to review and merge pull requests” — issues are welcomed, patches are not. To be fair to the project, the same tracker shows ten pull requests, all closed, and all authored by the vendor’s own maintainers or Dependabot (release automation, README updates and dependency bumps). So this is not a repository that never merges anything; it is one that does not merge outside work. If your organisation needs to patch its agent runtime, you are forking.

Who Should Use It in 2026

Embed the library if agent execution lives inside a product you ship and your stack is Go. The eight interfaces are the deliverable: you will replace the LLM adapter and probably the operation manager, keep the session store and the translator contract, and inherit crash-oriented session handling and idempotent inputs rather than writing them. Versioned sessions that error explicitly on resume and append-only storage with forking are genuinely rare in a three-week-old repository.

Shell out to the runner if you are evaluating the harness on your own tasks — that is the unreal-agent-runner path, and it is the fastest way to learn whether the turn-count advantage holds on your workload rather than on Terminal-Bench. It is also the honest way to find your own copy of issue #3 before it finds you.

Skip it if you pay a flat subscription and your only question is the monthly bill — token efficiency is then your provider’s problem, and the arithmetic above moves nothing — or if you need multi-agent delegation, a hosted sandbox or an IDE integration today.

The verdict is narrow and defensible: the architecture is more interesting than the headline statistic, and the headline statistic is more fragile than the architecture. Same model, same benchmark, same published pass rate, 39% less spend on the flagship row, with the direction repeated across three more suites — all of it self-run, on one model, with a summary chart the vendor itself called lazy and an open-issue list sitting on the async path. Evaluate it now; adopt it for Go-based agent infrastructure if you can cap, quota and kill what it starts; do not adopt it yet for anything that has to be boring.

How This Review Was Checked

The verification behind this article is source re-reading and arithmetic, and it is reproducible from a browser without any tooling:

1. The launch post was re-fetched on 12 October 2026 and every figure quoted here was matched against the live tables — the four totals rows, the token and turn columns, the Harbor job IDs, the xhigh effort label and the “These runs are not on Harbor” line on the ALE-CLI table.
2. The four relative-saving figures and the per-point costs were recomputed from those published totals in-session; the computations are shown inline above so any reader can check the division.
3. Repository metadata, the release list and the full issue list were read from the GitHub API on 12 October 2026; issue states in the risk table are taken from that response, not from an earlier draft.
4. The Hacker News thread was read through the Algolia item API: item 49805748, “Unreal Agent”, posted 2026-09-22T18:15:53Z, standing at 247 points with 125 comments on this read. Quotes attributed to commenters and to the vendor’s account in that thread come from that response.
5. The thinking-level mismatch, the Homebrew install path, the Docker invocation and the unrea1labs image spelling were read from the raw repository files listed below.

Not done, and not claimed: the harness was never installed, built or executed; no benchmark was re-run; no crash, cancellation or disk-growth behaviour was reproduced; and no search volume, traffic or revenue figure is asserted anywhere in this article. The demand picture for this topic was not measured here either. The fetch log for the checks above is an author-side private record and is not offered as a downloadable dataset; what is offered is the list of pages to open, below.

Source URLs checked on 12 October 2026:

• https://unreallabs.ai/blog/unreal-agent/
• https://github.com/unreallabsai/unreal-agent
• https://raw.githubusercontent.com/unreallabsai/unreal-agent/main/README.md
• https://raw.githubusercontent.com/unreallabsai/unreal-agent/main/harness/contextbuilder/prompts/preamble.md
• https://raw.githubusercontent.com/unreallabsai/unreal-agent/main/cmd/unreal-agent-runner/README.md
• https://raw.githubusercontent.com/unreallabsai/unreal-agent/main/benchmarks/harbor/README.md
• https://raw.githubusercontent.com/unreallabsai/unreal-agent/main/CONTRIBUTING.md
• https://raw.githubusercontent.com/unreallabsai/unreal-agent/main/Dockerfile
• https://raw.githubusercontent.com/unreallabsai/unreal-agent/main/go.mod
• https://raw.githubusercontent.com/unreallabsai/unreal-agent/main/Makefile
• https://raw.githubusercontent.com/unreallabsai/unreal-agent/main/.github/workflows/ci.yml
• https://raw.githubusercontent.com/unreallabsai/unreal-agent/main/internal/openaiapi/generate.py
• https://raw.githubusercontent.com/unreallabsai/unreal-agent/main/internal/anthropicapi/generate.py
• https://raw.githubusercontent.com/unreallabsai/unreal-agent/main/harness/coordinator/compaction_scheduling_test.go
• https://raw.githubusercontent.com/unreallabsai/unreal-agent/main/cmd/internal/agentrunner/resume_test.go
• https://raw.githubusercontent.com/unreallabsai/unreal-agent/main/harness/coordinator/stop_integration_test.go
• https://raw.githubusercontent.com/unreallabsai/unreal-agent/main/harness/coordinator/stop_test.go
• https://raw.githubusercontent.com/unreallabsai/unreal-agent/main/benchmarks/harbor/pyproject.toml
• https://api.github.com/repos/unreallabsai/unreal-agent
• https://api.github.com/repos/unreallabsai/unreal-agent/releases
• https://api.github.com/repos/unreallabsai/unreal-agent/contributors
• https://api.github.com/repos/unreallabsai/unreal-agent/commits?per_page=1
• https://api.github.com/repos/unreallabsai/unreal-agent/issues?state=all&per_page=100
• https://github.com/unreallabsai/unreal-agent/issues/3
• https://github.com/unreallabsai/unreal-agent/issues/5
• https://github.com/unreallabsai/unreal-agent/issues/10
• https://github.com/unreallabsai/unreal-agent/issues/11
• https://github.com/unreallabsai/unreal-agent/issues/13
• https://github.com/unreallabsai/unreal-agent/issues/14
• https://github.com/unreallabsai/unreal-agent/issues/15
• https://github.com/unreallabsai/unreal-agent/issues/16
• https://github.com/unreallabsai/unreal-agent/issues/22
• https://github.com/unreallabsai/unreal-agent/issues/23
• https://hn.algolia.com/api/v1/items/49805748
• https://news.ycombinator.com/item?id=49805748
• https://developers.openai.com/api/docs/guides/async-tool-calling
• https://inworld.ai/blog/introducing-unreal-ai-runtime-sdk

FAQ

Is Unreal Agent related to Epic’s Unreal Engine? No. Unreal Agent is an agent harness written in Go by Unreal Labs. Epic’s Unreal Engine is a C++ game engine from an unrelated vendor, and a third product, Inworld’s “Unreal AI Runtime”, is an Unreal Engine toolkit for AI characters. The name overlap is the project’s own discoverability problem: readers arriving from engine intent will bounce, and the project has to disambiguate in its first sentence, which is why this review does.

How much cheaper is Unreal Agent than Codex? On the vendor’s own Harbor runs at GPT-6 Astra xhigh, Terminal-Bench 4.0 came in at $1,428 against the Codex leaderboard row’s $2,350 for an identical published 57.9% pass rate — 1 − 1428/2350 = 39.2%. SWE-Atlas Codebase QnA was 28.2%, DeepSWE 1.1 was 16.3% and Agents’ Last Exam was 25.7%. Against the vendor’s own Pi runs the same rows are 21.8%, 9.4%, 13.7% and 17.2%. All of it is vendor-published and independently unreproduced.

Do async tool calls actually save tokens, or is that marketing? Partly structural, partly prompt-driven. A synchronous harness is not charged for the time it spends waiting, so “async saves the wait tokens” is not the mechanism. The measurable part is fewer model turns — 28 against Pi’s 44 per Terminal-Bench trial — and fewer input tokens per trial: 26.9% to 46.9% below Codex across the three Harbor rows, and 52.2% on the ALE-CLI row. The vendor’s own framing is “fewer model turns and fewer input tokens”, and the prompt tells the model to batch tool calls, so a like-for-like split between runtime and prompt is still unpublished.

Is Unreal Agent production-ready? Not on the evidence. It is at v0.3.1 with 2,190 stars, a six-entry contributor list, and eight open issues as of 12 October 2026 — including the unbounded disk-growth background-Bash loop (#3), the .env trust boundary (#5), and a 400 error that poisons a session when parallel fan-out overlaps (#11). Two async-adjacent issues (#15, #16) were closed on 6 October, so the direction is right. It is a serious runtime to evaluate and embed, not a platform to build policy on today.

Which providers and languages does it support? Go 1.27.0, MIT licensed. The provider is chosen at runtime from openai (default), openai-codex, openrouter, fireworks and ollama, and alternative implementations of the eight component interfaces are explicitly encouraged; the Harbor adapter additionally supports anthropic/<model> and fireworks_ai/<model>. No Python or TypeScript library binding is shipped, so non-Go stacks integrate through the runner executable or over the component interfaces; the Python in the tree is development tooling, not a client library — OpenAPI-to-Go generator scripts under internal/openaiapi/ and internal/anthropicapi/, PEP 723 inline scripts that shell out to oapi-codegen — plus the Harbor benchmark adapter under benchmarks/harbor/ with its own pyproject.toml, which is a benchmark integration rather than a client library.