Self-Hosting vs Inference APIs in 2026: The Real Cost Math

Self-hosting an LLM is cheaper than an inference API only when your monthly token volume clears one number: the break-even point where a GPU server's fixed cost, spread across your actual traffic, drops below per-token API pricing. Below that line, APIs win. Here is how to find your number.

Key takeaways

• The whole decision reduces to one formula: break-even tokens per month = fixed monthly GPU cost ÷ (API price per token − your marginal serving cost per token).
• Utilization is the multiplier everyone forgets: a GPU running at 20% load costs 5× more per served token than the same GPU fully loaded.
• The serving software is free: vLLM, Ollama, and similar stacks are open source. The GPU, and the ops hours to keep it alive, are the real bill.
• In the illustrative worked example below, a hypothetical $2,000/month GPU node against a hypothetical $0.50-per-million-token API breaks even near 4.4 billion tokens per month, or roughly 1,700 tokens per second sustained around the clock.
• If your traffic is spiky, seasonal, or modest, a hosted endpoint (including private, no-retention options) usually beats owning silicon; the crossover comes later than most teams expect.

Is self-hosting an LLM cheaper?

Sometimes, and the honest spread is wide. The two options have fundamentally different cost shapes, so the answer depends on where your workload sits on three axes: volume, utilization, and ops capacity.

An API is a pure variable cost: you pay per token, and when traffic stops, the bill stops. A self-hosted GPU is a pure fixed cost: the meter runs 24/7 whether you serve one token or one billion. Fixed costs beat variable costs only when you have enough volume to dilute them, which is why the entire debate collapses into a single computable threshold.

Three regimes emerge. Low or spiky volume almost always favors APIs. Sustained, predictable, high volume can favor self-hosting, sometimes decisively. And the crossover between them is arithmetic, not a vibe; the math is below.

What an inference API actually bills you for

API pricing is per token, usually with separate rates for input and output tokens, and sometimes discounted rates for cached context. That simplicity hides how much is bundled in: redundancy, autoscaling, model upgrades, uptime engineering, and the on-call rotation are all inside the per-token price.

The most underrated line item is idleness. An idle API client costs nothing. An idle GPU costs exactly the same as a busy one. For workloads with diurnal traffic (business-hours copilots, bursty agents, weekend-quiet dev tools), this asymmetry dominates the math.

For current per-token rates across hosted providers, check our running roundup of the cheapest LLM API options, and verify against provider pricing pages directly, since both sides of this market reprice frequently.

How much does a vLLM server cost?

vLLM itself is free, open-source serving software; the same is true of Ollama and most of the inference stack. The cost is everything around the software:

1. The GPU instance. This is the dominant line, typically the overwhelming majority of the monthly bill. You can rent on-demand (most expensive per hour, most flexible), reserve capacity with a commitment (cheaper, less flexible), or buy hardware and amortize it over its useful life while paying power and cooling.
2. The host around the GPU. CPU, RAM, fast NVMe storage for model weights, and bandwidth. Real money, but secondary.
3. Ops time. Driver updates, CUDA version wrangling, model migrations, throughput regressions, 3 a.m. outages. The line most spreadsheets omit entirely.
4. Utilization. A GPU is a fixed cost, so your cost per token is the fixed cost divided by tokens actually served. A half-loaded GPU doubles your per-token cost; a quarter-loaded one quadruples it.

That last point is where naive comparisons die. Sticker prices compare a fully-loaded GPU to an API; your traffic determines whether the GPU is ever fully loaded.

GPU vs API cost calculator: the break-even formula

Define four variables:

• F is the fixed monthly cost of self-hosting: GPU instance plus supporting hardware plus an honest allowance for ops hours.
• P is the blended API price per token for your workload (weight input and output rates by your real traffic mix).
• m is the marginal cost per token when self-hosting: power and bandwidth. Usually small, but include it; leaving it out flatters the self-hosting case.
• V is your actual monthly token volume.

Then:

Break-even volume B = F ÷ (P − m)

If V sits comfortably above B, self-hosting is a cost candidate. If V is below B, the API is cheaper, full stop. A useful conversion: divide B by 2.592 million (the seconds in 30 days) to get the average tokens-per-second you must sustain around the clock to break even. That reframing is sobering: break-even volumes that sound abstract become very concrete when expressed as throughput you must generate at 4 a.m. on a Sunday.

A worked example with clearly labeled assumptions

The numbers below are illustrative placeholders, not market quotes. Pull current GPU instance rates and per-token API prices from provider pricing pages and recompute; both move fast.

The math: B = $2,000 ÷ ($0.50 − $0.05) per million tokens ≈ 4.4 billion tokens per month. That is roughly 148 million tokens per day, or about 1,700 tokens per second sustained 24/7, around 34% utilization of the hypothetical 5,000 tok/s box.

Now the sensitivity, which matters more than the point estimate:

• If your blended API price doubles (heavier model mix, more output tokens), B halves to ~2.2 billion tokens.
• If you need high availability (and one GPU is one point of failure), F roughly doubles, and B doubles with it.
• If your traffic only keeps the box 15% loaded, your effective self-hosted cost per token is more than double the break-even assumption.

The levers are utilization, redundancy, and your real price mix, not the sticker price of the GPU.

The hidden line items that break naive cost math

On the self-hosting side: idle capacity during off-hours; the second node you eventually need for failover; ops hours that scale with model churn, since open-weight releases arrive constantly and each upgrade costs evaluation and migration time; plus observability, storage, and egress.

On the API side, the hidden costs are different but real: rate limits biting at your peak, long-context workloads compounding per-token spend, and data leaving your perimeter, which for some teams is a compliance cost, not just a philosophical one. Note that hosted providers' zero-retention and no-training promises are policy statements by those providers, not independently verified facts; self-hosting remains the only architecture where prompt data verifiably never leaves hardware you control.

The middle path: keep the privacy, drop the ops

The binary is a false one. Between "rent a GPU and run vLLM yourself" and "send everything to a frontier API" sits a growing tier: hosted endpoints for open-weight models, dedicated instances, and private endpoints with no-retention policies. Options include serverless open-weight hosts such as Together AI, Fireworks AI, and OpenRouter, as well as Morpheus, a decentralized inference marketplace that routes requests to independent operators. Evaluate them all on the same criteria: per-token price, model catalog, retention policy, and latency at your concurrency.

Because nearly everyone now exposes an OpenAI-compatible API, switching between self-hosted and hosted is a base-URL change, not a rewrite:

import os
from openai import OpenAI

client = OpenAI(
    # Self-hosted vLLM: "http://localhost:8000/v1"
    # Hosted provider: the provider's OpenAI-compatible endpoint
    base_url=os.environ["LLM_BASE_URL"],
    api_key=os.environ.get("LLM_API_KEY", "local"),
)

resp = client.chat.completions.create(
    model=os.environ.get("LLM_MODEL", "your-model-name"),
    messages=[{"role": "user", "content": "Hello"}],
)
print(resp.choices[0].message.content)

export LLM_BASE_URL="http://localhost:8000/v1"   # or your provider's endpoint
export LLM_API_KEY="local"

curl "$LLM_BASE_URL/chat/completions" \
  -H "Authorization: Bearer $LLM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "your-model-name",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

This is also the practical migration path for teams outgrowing a laptop: our guide to moving from Ollama to a private endpoint walks through keeping local-style privacy while shedding the ops burden. Many teams land on a hybrid: APIs and private endpoints for spiky traffic, self-hosted capacity for the steady baseline. That split is what the math usually recommends once you price both honestly.

Which side of the line are you on?

Run the checklist:

• Compute B with real quotes. Is your sustained monthly volume comfortably above it, not just at peaks?
• Is your traffic smooth enough to keep a GPU loaded, or diurnal and bursty?
• Does data need to stay on hardware you control, full stop?
• Do you serve custom fine-tunes that hosted catalogs do not carry?
• Do you have the ops hours to own a GPU, honestly accounted?

If most answers point at volume, control, and customization, self-host. If they point at elasticity and simplicity, stay on APIs. If they split, hybrid; the base-URL swap makes that cheap.

FAQ

Is self-hosting an LLM cheaper than using an API?

Only above your break-even token volume: the monthly traffic at which a GPU server's fixed cost, spread across real usage, drops below per-token API pricing. Below that volume, APIs are cheaper because you never pay for idle capacity.

How much does a vLLM server cost?

vLLM itself is free, open-source software. The real cost is the GPU instance (rented or amortized owned hardware), plus power, bandwidth, and ops time. Check current cloud GPU pricing pages for rates; they move frequently.

How do I calculate GPU vs API cost?

Break-even tokens per month = fixed monthly GPU cost ÷ (API price per token − your marginal serving cost per token). Compare your real monthly volume against it; sustained volume well above that line favors self-hosting.

When should you switch from an API to self-hosting?

When sustained volume sits comfortably above break-even, when data must stay on hardware you control, or when you serve custom fine-tunes. Spiky or modest traffic usually stays cheaper on APIs.