Terminal-Bench

Introduction — The Year AI Agent Benchmarks Broke If you have been following AI agent benchmarks in 2026, you have likely seen headline numbers that look too good to be true. A tool called BenchJack scored 100% on SWE-bench Verified, SWE-bench Pro, and Terminal-Bench without solving a single task — most runs never even called a language model. OpenAI retired SWE-bench Verified after discovering that 59.4% of hard failed tasks had broken test cases rejecting functionally correct patches. And the same frontier model scored 64.7% on one wrapper and 57.5% on another — a 7.2-point gap from scaffolding alone. This is the state of exploiting AI agent benchmarks in 2026: a system where the incentives to publish high scores have outpaced the rigor of the evaluations themselves. ...

Terminal-Bench 2.0 is the benchmark the DevOps and MLOps communities have needed for years. Unlike SWE-bench, which focuses narrowly on Python bug fixes in open-source repos, Terminal-Bench drops AI agents into a live terminal environment and asks them to do what senior engineers actually spend their days doing: compile unfamiliar codebases, configure servers, train models, write and debug scripts, and complete multi-step system administration tasks. As of May 2026, 39 models have been evaluated and the average score sits at 56.4% — a gap that reveals just how hard real terminal work is for even the most capable AI agents. ...

GPT-5.5 is OpenAI’s first fully retrained base model since GPT-4.5 — codenamed “Spud” internally — and it scores 82.7% on Terminal-Bench 2.0, making it the leading model for autonomous terminal-based coding tasks as of April 2026. If you’re deciding whether to migrate Codex pipelines or agentic coding workflows to GPT-5.5, this guide covers benchmarks, setup, computer use, and real workflow patterns. What Is GPT-5.5 and Why It’s a Big Deal for Developers GPT-5.5 is OpenAI’s most capable agentic model, launched April 23, 2026, to ChatGPT Plus, Pro, Business, and Enterprise subscribers. It is the first fully retrained base model since GPT-4.5 — internally codenamed “Spud” — rebuilt from the ground up for long-horizon agentic tasks rather than fine-tuned on top of GPT-5.4. Unlike incremental releases, GPT-5.5 changes the underlying model weights and reasoning patterns to prioritize terminal operations, computer use, and multi-step autonomous execution. On Terminal-Bench 2.0, it scores 82.7%, beating Claude Opus 4.7 (69.4%) by 13.3 percentage points and edging out Claude Mythos Preview (82.0%) in a near-statistical tie. On GDPval — a benchmark spanning 44 real-world occupations — it reaches 84.9%. For developers running coding agents, the practical implication is clear: GPT-5.5 handles bash-heavy autonomous workflows better than any prior model. However, on SWE-Bench Pro (real GitHub issue resolution), it scores 58.6% versus Claude Opus 4.7’s 64.3%, which means the model to choose depends heavily on whether your tasks live in the terminal or in production codebases. ...