Ollama Tool Definitions Are Not Free: The Measured Prompt-Token Cost of Every Tool You Register

You register twelve tools on a local agent, send a six-word message, and prompt_eval_count comes back at 764 instead of 18. Nothing is wrong: the definitions you sent are part of the prompt, and on a 4k default context that registry has already spent 746 of your 4096 tokens before the conversation starts — and it re-pays that cost on every turn. This article measures the cost instead of estimating it. One model, one fixed message, one local server, tool sets of 0, 1, 3, 6 and 12 definitions in a deliberately terse form, and sets of 1, 3 and 6 in a deliberately verbose form, three repeats per cell, with the raw request and response records published next to the script. ...

November 12, 2026 · 11 min · baeseokjae
Diagnostic sequence for a streaming tool call: preserve raw SSE, inspect reconstructed call boundaries, validate JSON and value types, and gate execution. This is not a new vLLM benchmark.

vLLM Tool-Call Streaming: Detecting Corrupted Arguments

A streaming tool call can reach your agent corrupted while every SSE frame on the wire is valid JSON. The damage shows up after reassembly: two calls fused into one, an argument string cut mid-token, or a number that became the string "3". The request returns HTTP 200, finish_reason says tool_calls, and an agent that trusts the payload executes the wrong thing. The practical question is not whether vLLM is “broken” in general; it is whether the corruption is in the server’s parser or in your client’s reassembly, and how to stop execution until that is settled. This article separates what the current primary reports actually establish, shows how to localize the fault with a raw SSE capture, and ships a small gate that refuses any call whose arguments do not round-trip against your declared schema. ...

October 28, 2026 · 10 min · baeseokjae
Best LLM for AI Agents 2026: GPT-5.5 vs Claude Opus 4.7 vs Gemini 3.1 Pro

Best LLM for AI Agents 2026: GPT-5.5 vs Claude Opus 4.7 vs Gemini 3.1 Pro on Tool Use and Reasoning

There is no single best LLM for AI agents in 2026 — Claude Opus 4.7 leads tool orchestration and code tasks, GPT-5.5 dominates terminal-style agentic workflows, and Gemini 3.1 Pro wins on context window and cost. Your model choice should follow your use case, not a global ranking. The LLM-for-Agents Landscape in 2026 (What Changed) The LLM-for-agents landscape changed fundamentally between 2024 and 2026, and the old question — “which model is smartest?” — has been replaced by a more precise one: “which model performs best on the specific agentic task I’m building?” As of May 2026, 31% of enterprises have at least one AI agent running in production, led by banking and insurance at 47%. Despite this momentum, 88% of enterprise AI agent pilots never reach production — with evaluation gaps (64%), governance friction (57%), and model reliability (51%) cited as the top blockers. The global enterprise AI agent spend is tracking a $1.4 trillion 2027 forecast, and the broader LLM market may reach $35.4 billion by 2030 at a 36.9% CAGR. What’s driving adoption is not a single breakthrough model, but an ecosystem shift: agentic frameworks (LangGraph, CrewAI, OpenAI Agents SDK), standardized tool protocols (MCP, function calling schemas), and multi-model routing that lets teams assign the right model to each task rather than betting everything on one provider. ...

May 14, 2026 · 12 min · baeseokjae