The cheapest way to cut an LLM API bill is not to pick one trick but to stack five or six levers that act on different parts of the same request: which model answers, how much of the prompt is re-processed, whether the call has to be synchronous, how long the output is, and how much conversation history rides along. Each lever multiplies against the cost the previous one already removed. This guide works through ten of them with the arithmetic shown, the current list prices they depend on (checked against vendor pricing pages on 2026-10-20), and an order in which to implement them.
Why LLM API Bills Keep Climbing Even as Token Prices Fall
Per-token prices have fallen far enough that the price spread between models is now the dominant cost variable, not the absolute rate of any one model. On the list prices published on OpenAI’s pricing page and Anthropic’s model pricing page, the eight models compared in Strategy 9 span $0.10 to $12.50 per million input tokens (gpt-6-luna and Claude Haiku 5.5 at the low end, gpt-5.6-cyber at the high end) — a 125× spread within that set on one basis and one date. Those pages also list higher rates outside that set, such as gpt-6-astra’s $20.00 per million input tokens for long context against $10.00 in its short-context tier. Output rates are a separate axis and run higher still: gpt-6-astra charges $10.00 per million input tokens and $50.00 per million output tokens, and DeepSeek charges $0.15 off-peak and $0.30 peak per million cache-miss input tokens for deepseek-flash against $0.60 and $1.20 for output.
That structure explains why a bill can rise while unit prices fall: the bill is the product of tokens sent and the rate of whichever model handled them, so growth in volume or a drift toward expensive models swamps a price cut. The waste is not mysterious, and it falls into three categories you can inspect in your own request logs:
- Redundant context: system prompts, document chunks, and conversation history re-sent on every request even when they have not changed, and re-billed at the full input rate when the provider’s cache is not engaged.
- Model over-provisioning: frontier models answering classification, extraction, and FAQ traffic that a $0.10–1.00/M-token model handles identically.
- Uncontrolled output: no
max_tokensceiling, verbose chain-of-thought left in production payloads, and no reuse of answers to identical queries.
Each of those is directly addressable with the strategies below, and most of them change the rate you pay rather than the volume you send.
Strategy 1 — Intelligent Model Routing (Save 40-70%)
Intelligent model routing means sending each request to the cheapest model that can answer it correctly instead of routing everything through one frontier model. The saving is arithmetic rather than a benchmark result: if 60–70% of your traffic is classification, entity extraction, structured parsing, or FAQ answering, and those requests move from a $10.00/M-input model such as gpt-6-astra to a $0.10–0.30/M-input model such as gpt-6-luna or deepseek-flash, the blended input cost falls by roughly the share of traffic you moved. Prices from OpenAI and DeepSeek, checked 2026-10-20.
| Approach | Complexity | Best For |
|---|---|---|
| Rule-based routing | Low | Well-defined task types |
| LLM classifier | Medium | Mixed workloads |
| LLM gateway (LiteLLM, Portkey) | Low | Teams wanting out-of-the-box routing |
| Embeddings-based routing | Medium-High | Semantic task classification |
Start with a rule-based router: tag request types (classification, summarization, code generation, open-ended reasoning) and assign a model tier to each. Run it in shadow mode against production traffic before you let it serve responses, then compare quality on a labelled sample of the traffic you intend to down-route. The rule set is usually where most of the saving lives, because the distribution of real traffic is far more skewed toward simple requests than teams expect.
Strategy 2 — Prompt Caching: Pay Once, Reuse Forever (Save up to 95%)
Prompt caching stores the processed state of a static prompt prefix so later requests that share that prefix are billed at a cached-input rate instead of the full input rate. The discount is large but not uniform across vendors, which is why the provider row you read matters more than the headline percentage. Anthropic charges cache hits at 5% of the input rate on Claude Sonnet 5.5 ($0.10 against $2.00 per million tokens) and 10% on Claude Haiku 4.5 ($0.10 against $1.00), while cache writes cost 125% of the input rate on a five-minute TTL — the figures on Anthropic’s pricing page. OpenAI charges cached input at one tenth of the input rate: $1.00 against $10.00 per million tokens on gpt-6-astra and $0.10 against $2.00 on gpt-6.1-sol, per OpenAI’s pricing page. DeepSeek goes further, billing cache hits at $0.003 per million tokens off-peak against a $0.15 cache-miss input rate on deepseek-flash.
Work the arithmetic on a real shape of workload. A 10,000-token system prompt sent 1,000 times a day is 10 million input tokens a day. At Claude Sonnet 5.5’s $2.00 per million input tokens that is $20.00 a day. If the prefix is cached and every request after the first is a cache hit at $0.10 per million, the same 10 million tokens cost $1.00 a day — about $19 a day, or roughly $570 a month, for one afternoon of prompt restructuring plus the one-off cache-write cost. The structural requirement is that static content (system instructions, retrieved documents, tool schemas) sits before the dynamic content, and that the prefix is long enough to be cacheable — OpenAI enables caching automatically for prompts of 1,024 tokens or longer.
Cached-input rates by provider (checked 2026-10-20):
| Provider | Cached-input rate | Note |
|---|---|---|
| Anthropic Claude | 5% of input on Sonnet 5.5; 10% on Haiku 4.5 | Cache writes cost 125% of input on the 5-minute TTL |
| OpenAI | 10% of input (gpt-6-astra $1.00 vs $10.00) | Automatic for prompts of 1,024 tokens or longer |
| DeepSeek | ~98% off (deepseek-flash $0.003 vs $0.15 off-peak) | Peak/off-peak pricing applies |
A cache hit rate near zero after the first request in a window usually means the prefix ordering broke or a timestamp or user ID crept in ahead of the static block. Log the provider’s cache-read field (cache_read_input_tokens on Anthropic, cached_tokens on OpenAI) per request and alert on the ratio, because this is one optimization where a silent regression is expensive and invisible in latency charts.
Strategy 3 — Semantic Caching at the Gateway Layer (Save up to 86% on repeat queries)
Semantic caching is a gateway layer that embeds the incoming query, compares it against previously answered queries, and returns a stored answer when the similarity exceeds a threshold — avoiding the model call entirely. AWS published a measured result for this pattern on Amazon ElastiCache with Amazon Bedrock: semantic caching reduced LLM inference cost by up to 86% and improved average end-to-end latency by up to 88% while maintaining 91% answer accuracy at a 0.75 similarity threshold, in the AWS database blog’s semantic-cache writeup. The saving scales with hit rate: at a 70% hit rate, 70% of your traffic stops consuming tokens at all, so the cost effect is the hit rate applied to the share of spend those queries represent.
Setup checklist:
- Deploy an embedding model (text-embedding-3-small is a common starting point) or use a managed gateway with a built-in semantic cache (LiteLLM, Portkey, Helicone).
- Set a similarity threshold conservatively at first — AWS reports its 91% accuracy figure at 0.75 — then tighten it where wrong answers are costly.
- Set a TTL based on how fast the underlying content changes.
- Monitor the false-positive rate: cached answers that are close but wrong are the failure mode, and they are invisible in cost dashboards.
Semantic caching and prompt caching are orthogonal: one skips the call, the other discounts the prefix of a call that still happens. Apply prompt caching to the traffic that misses the semantic cache.
Strategy 4 — Batch API Processing for Non-Real-Time Workloads (Save 50%)
Batch processing groups requests into an asynchronous job that the provider runs outside the synchronous path in exchange for a flat 50% discount. OpenAI’s Batch API documentation states a 50% cost discount compared with synchronous APIs and completion within 24 hours, and Anthropic’s pricing page documents the same 50% discount on both input and output tokens for the Message Batches API. Document summarization, nightly reporting, bulk classification, extraction from large corpora, and evaluation-set scoring all qualify.
The unit matters when you model the saving. Document classification is input-dominated, so price it on the input rate: 1 million input tokens a day on gpt-6-astra at its $10.00 per million input rate is $10.00 a day; the same volume in batch at the 50% discount is $5.00 a day, a saving of $5.00 a day or about $1,825 a year on that one pipeline. Using the model’s $50.00 output rate for that calculation would overstate the bill fivefold for a workload that barely generates output.
| Workload | Use Batch? |
|---|---|
| Chat assistant | No |
| Document ingestion | Yes |
| Nightly analytics | Yes |
| User-facing search | No |
| Evaluation pipelines | Yes |
| Content moderation queue | Yes (if async is acceptable) |
Both APIs take JSONL files of ordinary request objects, so the integration cost is mostly queue plumbing and idempotency rather than model work. The constraint to respect is the turnaround window: anything a user is waiting on cannot move to batch, and a retry policy still has to exist for failed batch items.
Strategy 5 — Prompt Compression and Token Optimization
Prompt compression reduces the tokens in a prompt before the request is sent, while preserving the information the model needs. The best-documented general method is Microsoft’s LLMLingua, which reports up to 20× compression with minimal performance loss on standard benchmarks in the LLMLingua paper (arXiv 2310.05736). Because compression acts on input tokens, the cost effect is proportional to how much of your bill is input and how much of that input is compressible retrieval context — a RAG query carrying 3,000–10,000 tokens of retrieved chunks is the natural target, and a short chat turn is not.
| Technique | Expected token reduction | Complexity |
|---|---|---|
| Whitespace and formatting cleanup | Small, single-digit to low-double-digit percent | Low |
| Instruction deduplication | Low-double-digit percent | Low |
| LLMLingua-style compression | Up to 20× compression on compressible context | Medium |
Treat the reduction figures as planning estimates and measure them on your own prompts: audit a sample of production requests, count what is actually removable, and only then decide whether a compression pipeline earns its latency cost. Compression also changes the prompt the model sees, so quality has to be evaluated on the compressed form, not assumed from the uncompressed one.
Strategy 6 — Context Window Management and Compaction
Context management is the discipline of controlling what history, documents, and memory enter each request. The cost mechanism is simple arithmetic: a 50-turn conversation at 200 tokens per turn adds 10,000 tokens of history to every subsequent request, so by the end of the conversation each new user message is billed alongside ten thousand tokens of context the model mostly does not need. Compaction — summarizing older turns while keeping recent ones verbatim — removes a proportional share of that, and dynamic retrieval replaces a full document dump with the handful of chunks a query actually needs.
Context management strategies:
- Sliding window: keep only the last N turns verbatim.
- Summarization compaction: collapse turns older than N into a short summary.
- Dynamic RAG: retrieve only the relevant chunks per query instead of injecting whole documents.
- Entity memory: extract key facts into a structured store instead of retaining raw turns.
Long context is not a discount. A model priced at $2.00 per million input tokens still charges $2.00 for every million tokens of history you resend, and gpt-6-astra’s long-context tier is billed at $20.00 per million input tokens against $10.00 in its short-context tier on OpenAI’s pricing page — so treating a large window as permission to send everything can double the rate as well as the volume.
Strategy 7 — Output Length Control and Truncation
Output length control caps how many tokens the model generates, through max_tokens limits, concision instructions, and structured output formats. It matters disproportionately because output is the expensive side of the meter on most current models: gpt-6-astra bills $50.00 per million output tokens against $10.00 input, gpt-6.1-sol $10.00 against $2.00, and Claude Sonnet 5.5 $10.00 against $2.00, per OpenAI and Anthropic.
Take a workload of 10,000 queries a day averaging 500 output tokens on a model priced at $10.00 per million output tokens: 5 million output tokens a day is $50.00 a day. Cutting the average to 200 tokens drops it to 2 million tokens a day, or $20.00 — a $30.00 daily saving, roughly $900 a month, from a change to the prompt and a parameter. The levers are explicit max_tokens limits per task type, instructions to answer in a fixed number of sentences, structured formats (JSON, bullet lists) that carry less connective prose, and stripping reasoning traces out of production payloads.
Output prices at major providers (per 1M tokens, checked 2026-10-20):
| Provider/Model | Output price |
|---|---|
| Claude Haiku 5.5 (prompts ≤100k tokens) | $0.50 |
| gpt-6-luna | $0.50 |
| deepseek-flash | $0.60 off-peak / $1.20 peak |
| Claude Sonnet 5.5 | $10.00 |
| gpt-6.1-sol | $10.00 |
| Claude Opus 5.5 | $20.00 |
| gpt-6-astra | $50.00 |
| gpt-5.6-cyber | $75.00 |
For agentic workflows that genuinely need long reasoning, separate the reasoning step from the action step: let a cheaper model or an extended-thinking call do the thinking, and have the production response be a constrained, structured payload. Note that on models with a long-context tier the output rate can rise too — gpt-6-astra long context is billed at $75.00 per million output tokens against $50.00 short context.
Strategy 8 — Model Distillation: Train Once, Infer Cheaply
Distillation uses a large teacher model to label data, then trains a smaller student to reproduce the teacher’s behaviour on your task distribution. Once served, the student runs on cheaper hardware or a cheaper per-token rate, so the saving is structural rather than a discount you have to keep re-earning. The economics work when the task is high-volume and well-defined (extraction, classification, structured generation), stable enough that the student does not drift out of date, and you can produce a few thousand high-quality teacher examples to train and evaluate on.
| Condition | Proceed? |
|---|---|
| Volume above roughly 100K queries/month | Yes |
| Task is well-defined and stable | Yes |
| Task changes frequently | No |
| Team has ML infrastructure | Yes |
| Team is API-only | No |
The upfront cost is real: compute, evaluation tooling, and engineering time for a first project, plus an ongoing responsibility to detect when production inputs drift away from the training distribution. Treat any multiplier quoted for distillation as an assumption to be measured on your own task, because the reduction depends on the price gap between the teacher and the student you can actually serve, and on how much quality you are willing to trade.
Strategy 9 — Provider Diversification and Cost-Based Switching
Provider diversification spreads workloads across models and vendors by cost, capability, and availability instead of committing everything to one. The spread justifies it: among the eight models in the table below, whose standard short-context input rates come from OpenAI’s pricing page, Anthropic’s pricing page and DeepSeek’s pricing page, the cheapest is $0.10 per million tokens and the most expensive is $12.50 — about 125×. That figure is scoped to those eight models on one date and one tier, and it excludes rates listed outside the table: gpt-6-astra’s long-context input, for example, is $20.00 per million tokens. It also holds only because both ends are input rates; mixing a cheapest-input figure with a most-expensive-output figure would inflate it to a meaningless number, since output rates are a separate axis.
Standard input prices (per 1M tokens, checked 2026-10-20):
| Model | Input price | Best for |
|---|---|---|
| gpt-6-luna | $0.10 | High-volume simple tasks |
| Claude Haiku 5.5 (≤100k prompts) | $0.10 | Classification, extraction, routing |
| deepseek-flash | $0.15 off-peak / $0.30 peak | High-volume simple tasks |
| Claude Sonnet 5.5 | $2.00 | General-purpose reasoning |
| gpt-6.1-sol | $2.00 | General-purpose reasoning |
| Claude Opus 5.5 | $4.00 | Hard reasoning |
| gpt-6-astra | $10.00 | Frontier capability only |
| gpt-5.6-cyber | $12.50 | Specialised security workloads |
Cost-based switching goes a step further: an LLM gateway watches prices and moves traffic to whichever provider is cheapest for a capability tier, and fails over when a provider returns errors or slows down. LLM gateways (LiteLLM, Portkey, OpenRouter) handle the multi-provider abstraction, unified billing, and fallback logic. For most teams, adopting a gateway is a one-day implementation that unlocks multi-provider routing without custom engineering.
There is one further option that gateways do not address, because it changes the billing model instead of the rate: running the model on hardware you own. Open-source inference engines such as Magnitude, a self-optimizing local inference engine describe themselves as “free, private, open source: no token costs” (project README, documentation). That removes the token line item but not the cost—you trade per-token billing for hardware, electricity and operations, so it only makes sense for steady, high-volume workloads that keep the machine busy. It is also worth reading the project’s headline number for what it measures: “over 2x faster than Ollama” is a throughput comparison averaged over 11 popular local models on Mac and NVIDIA (1.32x prefill and 2.19x decode on Mac; 1.34x and 2.17x on NVIDIA), which says nothing about your API bill and is not comparable to the savings percentages in this guide.
Strategy 10 — Real-Time LLM Cost Monitoring and Alerting
Cost monitoring instruments each LLM call with per-request cost tracking, aggregates it into dashboards, and alerts when spending moves outside a threshold. Without it, a runaway agent loop, a retry storm, or an accidental context blow-up can bill thousands of dollars before anyone notices, and none of the earlier strategies can be verified as working. Tools such as Langfuse, Helicone, OpenObserve, and provider-native dashboards now expose per-request cost breakdowns, model-level aggregation, cache hit rates, and anomaly detection.
Key metrics to track:
| Metric | Why it matters |
|---|---|
| Cost per request | Spot routing failures |
| Cache hit rate | Validate that caching is actually engaging |
| Output token ratio | Detect over-generation |
| Model distribution | Confirm routing logic is active |
| Cost per user | Identify high-cost usage patterns |
The minimum viable setup records, for every call, the model ID, input and output token counts, the cached-token count, and the computed cost from the price table you are actually billed against. Alert on cost per hour crossing a threshold, single-request outliers (a proxy for runaway context), and daily spend exceeding forecast by more than 20%. Wire a cost regression check into CI so a prompt change that adds 30% tokens shows up in code review instead of on the invoice.
How to Stack These Strategies
Stacking compounds because each lever applies to the cost the previous one left behind: routing changes the rate, caching discounts the prefix, batching halves what remains for async work, output control shrinks the expensive side of the meter, and context management removes tokens that never needed to be sent. The order below maximizes saving per engineering hour, with the cheap structural wins first.
| Week | Strategy | Expected effect |
|---|---|---|
| Week 1 | Model routing | Blended input cost falls by the share of traffic moved down-tier |
| Week 1-2 | Prompt caching | Cached prefixes billed at 5-10% of the input rate |
| Week 2 | Batch API for async workloads | 50% on eligible requests |
| Week 2-3 | Output length control | Output spend falls with average output length |
| Week 3-4 | Context management | History and retrieval tokens stop compounding per turn |
| Month 2 | Semantic caching | Repeat queries skip the model call entirely |
| Month 2-3 | Token compression | Input-token reduction on context-heavy pipelines |
| Month 3+ | Distillation (if high volume) | Structural rate reduction, at training cost |
The honest way to forecast a combined result is to model your own traffic: take your request mix, apply the routing split you can defend on quality, apply the cache hit rate you actually observe, and apply the batch share your latency budget allows. That is also the only way to know whether a 70% overall reduction is reachable for you — the levers multiply, but so do their preconditions, and a workload with no repeated prefix and no async traffic will not collect the caching or batching savings no matter how the guide is sequenced.
Frequently Asked Questions
What’s the single highest-ROI LLM cost reduction strategy for most teams? Model routing, because the price spread between tiers is enormous — $0.10 to $0.30 per million input tokens at the cheap end against $10.00 to $12.50 at the frontier end, per OpenAI’s pricing page checked 2026-10-20 — and the implementation is a rule set rather than infrastructure. If you only do one thing, move the traffic you can prove is simple onto a cheap model and measure quality on a labelled sample first.
Does switching to a cheaper model hurt output quality? For well-defined tasks such as classification, structured extraction, and FAQ answering, the cheap tier is often indistinguishable in evaluation. The gaps that matter appear in open-ended reasoning, long instruction-following, and creative work. The answer is routing rather than replacement: keep the frontier model for the requests that need it, and evaluate on a sample of the specific traffic you intend to move down before you commit the rule.
How do I measure whether prompt caching is actually working?
Read the cache fields the provider returns — cache_read_input_tokens on Anthropic, cached_tokens on OpenAI — and compute the cached share of input tokens per request. A structured prompt with a long static prefix should show cache reads on every request after the first within the cache window. If the ratio sits near zero, check whether dynamic content such as a timestamp, session ID, or per-user preamble has crept in ahead of the static block.
Is model distillation worth it for a team without ML infrastructure? Not as a first move. Distillation needs training and evaluation infrastructure plus a way to detect distribution drift, and the API-level strategies in this guide require none of that. Revisit it when the task is high-volume and stable, the API-level levers are exhausted, and you have engineering capacity to own a model rather than an API call.
What LLM gateway should I use to implement these strategies? LiteLLM is the most common open-source option and covers multi-provider routing, semantic caching, batching, and observability in a self-hosted deployment. Portkey and Helicone are managed alternatives with less operational overhead. OpenRouter is useful if you want a hosted multi-provider endpoint without running a gateway. For teams already on AWS, Bedrock’s gateway layer covers provider abstraction with native IAM integration. The choice turns on hosting preference and whether data privacy or reduced operations matters more.
