How to Run gpt-oss-120b via API in 2026 (Apache 2.0)

You can run gpt-oss-120b behind any OpenAI-compatible API by changing two lines (the base URL and API key) or self-host the weights on a single 80 GB GPU. OpenAI's 117B-parameter mixture-of-experts model ships under Apache 2.0, so you pay no per-token license fees either way.

Key takeaways

• Two integration lines, base_url and api_key, move any OpenAI SDK client to a gpt-oss-120b endpoint; the model id is usually gpt-oss-120b or openai/gpt-oss-120b.
• Self-hosting floor: OpenAI says the MXFP4-quantized weights (117B total, ~5.1B active per token) fit on one 80 GB GPU; community quantizations go lower at some quality cost.
• License: Apache 2.0. Commercial use, modification, and redistribution are permitted with zero usage restrictions in the license text; your provider's acceptable-use policy still applies on managed hosts.
• Reasoning effort (low / medium / high) is the main quality-vs-latency dial on gpt-oss models.
• Managed gpt-oss-120b APIs typically price at a fraction of frontier-API rates. Compare per-token prices on each provider's pricing page, current as of 2026-08-20.

What is gpt-oss-120b?

gpt-oss-120b is the larger model in OpenAI's open-weights line, released alongside the smaller gpt-oss-20b. In its announcement, OpenAI described the pair as its first open-weight language models since GPT-2. The release put Apache 2.0-licensed weights on Hugging Face for anyone to download, inspect, fine-tune, and serve.

Per the model card, gpt-oss-120b is a mixture-of-experts transformer with 117 billion total parameters but only about 5.1 billion active per token. That sparsity is why a model with "120b" in the name is cheap to serve relative to dense models of similar total size: each token only touches a small slice of the network. The card also lists a long native context window (128k tokens), though many API providers cap context lower, so check the per-provider limit.

Two design details matter in practice. First, gpt-oss models were trained in OpenAI's "harmony" response format, which structures reasoning, tool calls, and final answers. Any serious serving stack applies this chat template for you, but raw-completions users must format it themselves. Second, the model exposes a reasoning-effort setting (low, medium, high) that trades latency and token spend against answer quality. DeAI's model profiles track gpt-oss-120b alongside the rest of the open-weight field if you want a side-by-side spec view.

What are gpt-oss-120b's hosting requirements?

For self-hosting, OpenAI says the default MXFP4 quantization fits on a single 80 GB GPU (H100 or A100 80 GB class). Plan for on the order of 60 GB of disk for the weights, plus VRAM headroom for the KV cache. Long-context, high-concurrency production deployments will want more than the bare floor, either via a second GPU with tensor parallelism or by capping context and batch size.

If one 80 GB card is out of reach, you have three realistic options:

• Community quantizations. Lower-bit builds shrink VRAM requirements further, at some quality cost. Evaluate on your own prompts before committing.
• gpt-oss-20b. The smaller sibling targets single consumer-grade GPUs and is a reasonable stand-in during development.
• CPU offload. Possible with llama.cpp-style stacks, but generally too slow for interactive workloads at the 120b size.

For serving software, vLLM is the common choice for throughput-oriented deployments; Ollama is the simplest single-node path. Both handle the harmony chat template and expose an OpenAI-compatible endpoint.

Which providers host gpt-oss-120b?

If you don't want to own GPUs, managed inference is the faster path. As of 2026-08-20, gpt-oss-120b is broadly available across serverless providers such as Together, Fireworks, and Groq; aggregators like OpenRouter that route across multiple backends; privacy-focused providers such as Venice; and Morpheus, a decentralized inference marketplace. Lineups shift frequently, so treat this as a starting list, not a ranking.

Evaluate every option on identical criteria:

1. Exact model id and quantization served. Some hosts run the default MXFP4 build, others serve different quants.
2. Context cap and throughput limits actually enforced on your tier.
3. Price per million tokens on the provider's published pricing page.
4. Region and latency relative to your users.
5. Data-retention policy. Note that zero-retention and similar privacy promises are policy statements by the provider, not independently verified facts. Weigh them against your compliance requirements.
6. Status page and historical uptime.

DeAI's roundup of the best open-source LLM APIs applies this checklist across the major hosts.

How do you call gpt-oss-120b from code?

Nearly every host exposes an OpenAI-compatible API, so integration is a base-URL swap against the official SDK:

from openai import OpenAI

client = OpenAI(
    base_url="https://api.your-provider.example/v1",  # your provider's endpoint
    api_key="YOUR_API_KEY",
)

response = client.chat.completions.create(
    model="gpt-oss-120b",  # some providers use "openai/gpt-oss-120b"
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain mixture-of-experts routing in one paragraph."},
    ],
    # reasoning_effort="medium",  # low | medium | high, if your provider supports it
)

print(response.choices[0].message.content)

The equivalent curl call:

curl https://api.your-provider.example/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-oss-120b",
    "messages": [
      {"role": "user", "content": "Explain mixture-of-experts routing in one paragraph."}
    ]
  }'

Practical notes: the model id string varies by provider, so copy it from their docs. reasoning_effort support also varies. Where it isn't exposed as a parameter, the model card describes setting the effort level in the system prompt instead. Streaming, retries, and error handling behave exactly as they do for any other OpenAI-compatible backend, which is the point: gpt-oss-120b can slot into existing OpenAI client code without a rewrite.

How do you self-host gpt-oss-120b?

With vLLM on a suitable GPU node:

# Requires roughly 80 GB of VRAM for the default quantization
vllm serve openai/gpt-oss-120b --port 8000

With Ollama for a simpler single-machine setup:

ollama pull gpt-oss:120b
ollama serve  # exposes an OpenAI-compatible API at http://localhost:11434/v1

Then point the same Python client at your own endpoint (base_url="http://localhost:8000/v1" for vLLM) with any placeholder API key unless you've configured authentication. The client code doesn't change; only the URL does.

Two production caveats. First, if you bypass a chat-template-aware server and call raw completions, you must render the harmony format yourself. OpenAI's reference library is on GitHub, but most teams should simply use vLLM or Ollama and let them handle it. Second, never expose an unauthenticated inference port to the network: put a reverse proxy with TLS and auth in front, tune vLLM's batching and max-num-seqs for your traffic, and monitor GPU utilization and queue depth from day one.

What does the gpt-oss usage policy actually say?

This is where confusion is common, so separate the three layers:

1. The license. gpt-oss weights are Apache 2.0. The license permits commercial use, modification, and redistribution, includes a patent grant, has no copyleft and, unlike some other "open" model licenses, no field-of-use restrictions or user-count clauses. It does not grant rights to OpenAI's trademarks.
2. OpenAI's usage policies. These are contractual terms that bind users of OpenAI's own hosted services. If you self-host the weights, your relationship with the model is governed by the Apache 2.0 license, not OpenAI's service terms. Many teams still treat OpenAI's published policies as a sensible baseline for their own acceptable-use rules.
3. Your provider's terms. If you use a managed host, whether centralized or a decentralized inference marketplace, that platform's acceptable-use policy applies to your traffic, and policies differ. Read them before you build.

Finally, license freedom is not a liability shield: the laws of your jurisdiction and your industry's regulations apply to whatever you build regardless of where the weights came from. None of this is legal advice.

gpt-oss-120b vs GPT-5.5: which should you pick?

GPT-5.5 is OpenAI's closed, frontier-tier model; gpt-oss-120b is the open-weight line. Without inventing numbers, the honest framing is this: frontier proprietary models generally lead on the hardest reasoning, coding, and agentic tasks, and you should verify the current gap on public leaderboards and, better, on an eval set drawn from your own workload.

Choose gpt-oss-120b when you need downloadable weights (fine-tuning, quantization, auditing), data control (self-hosted inference means prompts never leave your infrastructure), predictable unit costs at high volume, or portability across providers. Choose GPT-5.5 when peak capability matters more than control and you'd rather own zero operations. Many production teams run both: a frontier model for the hardest or most sensitive reasoning steps, gpt-oss-120b, typically at a fraction of frontier-API pricing, for high-volume routine traffic, with a router in between. The OpenAI-compatible interface shared by both makes that split a configuration detail rather than an engineering project.

FAQ

What is the gpt-oss usage policy?

gpt-oss weights ship under Apache 2.0, which itself imposes no usage restrictions. OpenAI's usage policies bind OpenAI-hosted services; third-party and decentralized hosts apply their own acceptable-use rules, so check your provider's terms.

How does gpt-oss-120b compare to GPT-5.5?

GPT-5.5 is OpenAI's closed frontier model and is generally stronger on the hardest reasoning and coding tasks. gpt-oss-120b trades some peak capability for downloadable weights, fine-tuning freedom, data control, and typically a fraction of frontier-API pricing.

What are gpt-oss hosting requirements?

OpenAI's model card describes a 117B-parameter MoE with roughly 5.1B active parameters per token; OpenAI says the MXFP4-quantized weights fit on one 80 GB GPU. Community quantizations and CPU offload trim hardware further at a speed or quality cost.

Can you use gpt-oss-120b commercially?

Yes. Apache 2.0 permits commercial use, modification, and redistribution without copyleft or per-token license fees. The license does not grant OpenAI trademark rights, and your hosting provider's own terms still apply.