# DeAI News — full content

> AI you control — open models, private inference, and the networks that run them. Independent daily coverage plus verification-graded provider trackers.


---

# Abliterated and Uncensored AI Models, Explained (2026)

*By DeAI Newsroom · 2026-08-24 · 9 min read*

Canonical: https://www.deai.org/news/abliterated-uncensored-models-explained

> **Summary:** Abliteration removes an LLM's refusal behavior by editing a single direction in its activation space — no retraining required. 'Uncensored' is the broader umbrella, covering abliterated checkpoints, fine-tunes on de-refused data, and steered base models.

Abliterated models are open-weight LLMs whose refusal behavior is removed at the weight level by ablating a single direction in the model's activation space, with no retraining and no new data. "Uncensored" is the broader label for any model modified to answer prompts the original would refuse. Each technique changes the model in a different way, and the quality cost depends on which one you use.

## Key takeaways

- Published research found that refusal in chat-tuned LLMs is mediated by a **single direction** in activation space; abliteration edits the weights so that direction can no longer be expressed, suppressing refusals without any training run.
- "Uncensored" describes at least **three different artifacts**: de-refused fine-tunes, abliterated checkpoints, and steered base models. They behave differently and fail differently.
- Quality loss is technique-dependent, not inevitable: uncensored fine-tunes usually track their base model on standard benchmarks, while aggressive abliteration can nick adjacent behaviors.
- Standard benchmarks measure capability, not refusal. Evaluate **both axes** (harmful-prompt compliance and false refusals on benign prompts) before deploying anything.
- Serving choice matters as much as model choice: retention policies, catalog availability, and pricing vary across self-hosting, serverless providers, and decentralized inference marketplaces.

## What is abliteration?

Abliteration is a weight-editing technique that suppresses a language model's refusal behavior without retraining it. The term was coined by the researcher FailSpy in 2023, building on a simple observation: when an instruction-tuned model refuses a request, that refusal shows up as a consistent pattern in its internal activations.

A 2024 paper, ["Refusal in Language Models Is Mediated by a Single Direction"](https://arxiv.org/abs/2406.11717), formalized the idea across many popular chat models. The researchers found that the difference between how a model processes harmful versus harmless prompts concentrates into a single direction in the model's residual stream. Maxime Labonne's tutorial ["Uncensor any LLM with abliteration"](https://huggingface.co/blog/mlabonne/abliteration) then turned the finding into a reproducible recipe that most practitioners still follow.

The mechanics, in plain terms:

1. Run a set of harmful prompts and a set of harmless prompts through the model and record the activations at each layer.
2. Compute the direction that best separates the two sets: the "refusal direction."
3. Orthogonalize the model's weight matrices against that direction, so the network can no longer write to it.

The output is a new checkpoint that behaves like the original model but declines far fewer requests. Early examples are on [FailSpy's Hugging Face profile](https://huggingface.co/failspy). Three properties matter for practitioners:

- **It's data-free.** No training set, no fine-tuning run, no GPU cluster. The compute involved is trivial next to training.
- **It's model-specific.** The refusal direction must be computed per checkpoint; you can't transplant one model's edit to another.
- **It removes a behavior; it adds nothing.** An abliterated model knows exactly what the base model knew. It simply refuses less.

## What does "uncensored LLM" actually mean?

"Uncensored" is an umbrella term, and it covers at least three meaningfully different things:

**1. De-refused fine-tunes.** The oldest approach. You fine-tune a base or instruct model on instruction data where refusal completions ("I'm sorry, but as an AI language model…") have been filtered out or replaced with compliant answers. The [Dolphin series](https://huggingface.co/cognitivecomputations) from Cognitive Computations is the best-known lineage: datasets curated so the model learns to respond to the prompt rather than to a policy layer.

**2. Abliterated checkpoints.** The weight-edit approach described above. No examples, just geometry.

**3. Steered base models.** The weakest sense of the term: a base (non-instruct) model, or an instruct model wrapped in a system prompt that pushes it toward compliance. Nothing about the model changed; the "uncensoring" lives in the prompt and evaporates if the serving stack overrides it.

Two caveats keep this honest. First, uncensored is a **spectrum, not a binary**. Almost no released model answers literally everything; most "uncensored" checkpoints still decline some categories, and different derivatives draw the line in different places. Second, uncensored is not the same as jailbroken. A jailbreak manipulates a model at inference time, one conversation at a time; the models above have the behavior changed in the artifact itself.

## How is abliteration different from an uncensored fine-tune?

| | Abliteration | Uncensored fine-tune |
|---|---|---|
| Mechanism | Weight orthogonalization against the refusal direction | Gradient training on de-refused examples |
| Data needed | A small prompt set to locate the direction | A curated instruction dataset |
| Compute | Minimal; no training run | A real fine-tuning run |
| What changes | Suppresses one behavior globally | Reshapes behavior by example, can add style or skills |
| Typical failure mode | Collateral damage if cut too aggressively | Dataset biases, quality tied to data quality |
| Reversibility | Permanent checkpoint change | Permanent checkpoint change |

The two compose, and many popular "uncensored" releases are both: a de-refused fine-tune that is then abliterated for good measure. If your goal is simply "the base model, but it stops lecturing me," abliteration is the surgical option. If you also want the model to answer in a particular style or handle refusals gracefully rather than bluntly, a fine-tune gives you that control.

## Do uncensored models lose quality?

Not inherently. The answer depends on the technique.

**Fine-tunes** tend to land close to their base model on standard capability benchmarks, because de-refusal filtering removes examples rather than adding noise. The quality ceiling is set by the dataset: a well-curated uncensored fine-tune can match or exceed its base on the tasks its data covers, and a sloppy one degrades for reasons that have nothing to do with censorship.

**Abliteration** is targeted in theory but blunt at the edges in practice. The refusal direction is computed from a finite prompt set, and practitioners report that aggressive orthogonalization can nick adjacent behaviors: over-compliance with everything, flatter instruction-following, or incoherence on unusual inputs. How much this shows up varies by model and by implementation, which is why "abliterated" releases from different authors are not interchangeable.

The deeper problem is an **eval blind spot**. Standard public benchmarks measure capability (math, coding, knowledge) and say nothing about refusal behavior. A model can look identical to its base on every leaderboard while its refusal profile changed completely, which is the entire point of the modification. So "it benchmarks the same" tells you the capability survived; it tells you nothing about whether the model now complies with things it shouldn't, or still refuses benign requests it should answer. DeAI's [refusal-index methodology](/refusals) is designed around exactly this gap: it scores refusal behavior across harm categories alongside benign control prompts, so both over-compliance and false refusals become visible on one axis.

## How do you evaluate an uncensored model before deploying it?

A practical checklist:

1. **Start from the base model's published evals**, then ask whether the derivative publishes its own. Many don't, and silence is information.
2. **Run a refusal probe.** A fixed prompt set spanning clearly harmful, dual-use, and benign-but-sensitive categories. Measure two numbers: compliance on the harmful set and the false-refusal rate on the benign set. Frameworks like [HarmBench](https://arxiv.org/abs/2402.04249) exist to standardize this kind of red-team measurement.
3. **Regression-test on your own tasks.** A small golden set from your actual workload catches degradation that public benchmarks miss.
4. **Check the license.** Permissive licenses such as Apache 2.0 generally allow modification and redistribution; gated licenses (the [Llama 3.1 model card](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct) is a representative example) attach acceptable-use policies that restrict what you may do with outputs, modified or not.
5. **Decide where it runs.** A model's behavior is fixed at the checkpoint, but your exposure isn't: provider retention policies, catalog availability, and pricing all vary. DeAI's [comparison of uncensored-capable API providers](/best-uncensored-ai-api) tracks that side of the decision.

## Where can you run abliterated and uncensored models?

The options below are assessed on identical criteria: model availability, price per token, stated retention policy, and terms of service.

- **Self-hosting** with llama.cpp, Ollama, or vLLM. Full control, and the retention question disappears because no third party sees the traffic. You own the hardware and the ops burden.
- **Serverless inference providers** such as Together, Fireworks, DeepInfra, or OpenRouter. Catalogs vary, and some providers decline to host abliterated variants. Check each provider's model list and acceptable-use policy. Retention promises on any hosted API are policy statements, not independently verified facts, unless an audit says otherwise.
- **Decentralized inference marketplaces.** Morpheus is one such marketplace, routing requests to independent operators rather than a single company's servers. The same four criteria apply; the difference is architectural, not a free pass on due diligence.
- **Privacy-positioned providers.** Venice, for example, markets private inference. Treat zero-retention language as a positioning claim unless it is independently audited.

For a model you can run on your own hardware, self-hosting settles most of these questions by default. For burst capacity or larger checkpoints, compare providers on the same four criteria rather than on marketing labels.

## Who uses uncensored models, and what are the risks?

The legitimate demand is larger than the edgy branding suggests. Fiction writers need models that will depict violence and morally difficult characters. Security researchers and red teams need models that will engage with attack techniques to study defenses, the same motivation behind standardized tools like HarmBench. Medical, legal, and journalism users hit false refusals on benign-but-sensitive questions constantly. And alignment researchers need to manipulate refusal behavior to understand it; abliteration itself is a product of that research.

The risks are real and attach to use, not to the model file. Facilitating harm carries liability regardless of which checkpoint produced the output. Gated licenses and platform terms of service still apply to derivatives. And an uncensored model deployed in a product without a policy layer of your own is an incident waiting for a prompt. The technique is neutral; the deployment isn't.

## FAQ

### What is abliteration?

Abliteration is a weight-editing technique that suppresses an LLM's refusal behavior by finding the single activation direction linked to refusal and orthogonalizing the model's weights against it, with no retraining or new data required.

### What does "uncensored LLM" mean?

An uncensored LLM is an open-weight model modified to answer prompts the original model would refuse, whether by fine-tuning on de-refused data, by abliteration, or both. It's a spectrum rather than a binary: most uncensored models still decline some categories.

### Do uncensored models lose quality?

Not inherently. Uncensored fine-tunes typically score close to their base models on standard capability benchmarks. Abliteration is targeted but can degrade behavior if applied aggressively. Judge both capability evals and refusal behavior, including false refusals on benign prompts.

### Is abliteration the same as jailbreaking?

No. A jailbreak tricks a model at inference time with crafted prompts, one conversation at a time. Abliteration edits the weights, so reduced refusal applies to every prompt with no special input. Jailbreaks can be patched at the prompt level; abliteration is a permanent checkpoint change.

## FAQ

**What is abliteration?**

Abliteration is a weight-editing technique that suppresses an LLM's refusal behavior by finding the single activation direction linked to refusal and orthogonalizing the model's weights against it — no retraining or new data required.

**What does 'uncensored LLM' mean?**

An uncensored LLM is an open-weight model modified — by fine-tuning on de-refused data, by abliteration, or both — to answer prompts the original model would refuse. It's a spectrum rather than a binary: most uncensored models still decline some categories.

**Do uncensored models lose quality?**

Not inherently. Uncensored fine-tunes typically score close to their base models on standard capability benchmarks. Abliteration is targeted but can degrade behavior if applied aggressively. Judge both capability evals and refusal behavior — including false refusals on benign prompts.

**Is abliteration the same as jailbreaking?**

No. A jailbreak tricks a model at inference time with crafted prompts, one conversation at a time. Abliteration edits the weights, so reduced refusal applies to every prompt with no special input. Jailbreaks can be patched at the prompt level; abliteration is a permanent checkpoint change.

## Sources

- [Refusal in Language Models Is Mediated by a Single Direction](https://arxiv.org/abs/2406.11717) — arXiv
- [Uncensor Any LLM With Abliteration](https://huggingface.co/blog/mlabonne/abliteration) — Hugging Face
- [FailSpy — Hugging Face Profile](https://huggingface.co/failspy) — Hugging Face
- [Cognitive Computations (Dolphin Models)](https://huggingface.co/cognitivecomputations) — Hugging Face
- [HarmBench: Measuring Automated Red Teaming and Robust Refusal](https://arxiv.org/abs/2402.04249) — arXiv
- [Meta Llama 3.1 8B Instruct Model Card](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct) — Meta


---

# The Best LLM APIs for AI Agents in 2026 (Tool Use, Filters, Priced)

*By DeAI Newsroom · 2026-08-24 · 9 min read*

Canonical: https://www.deai.org/news/best-llm-api-for-ai-agents

> **Summary:** Picking an LLM API for AI agents is about finishing multi-step tool-use loops, not topping chat benchmarks. This roundup compares seven providers — an aggregator, hosted open-weight platforms, and a decentralized inference marketplace — on function-calling reliability, filter.

The best LLM API for AI agents in 2026 is the one that finishes your tool-use loops, not the one topping a chat benchmark. A single malformed function call or mid-loop refusal can sink a 30-step run. Below, we compare seven providers on tool calling, filter behavior, privacy posture, and cost per completed task.

## Key takeaways

- Tool-call reliability beats leaderboard rank: one invalid JSON argument in a 20-step loop fails the entire task.
- All seven providers in this roundup expose OpenAI-compatible endpoints, so switching is a base-URL swap, roughly two lines of code.
- Filter behavior is the hidden differentiator: a single mid-loop refusal can zero out an otherwise cheap run.
- Cost per completed task, retries included, is the only pricing number that matters; per-token prices mislead agent builders.
- Privacy and retention claims are provider policy statements, not independently verified facts. Treat them as such.

## What makes an LLM API good for AI agents?

An agent is not a chatbot. A chatbot makes one model call; an agent makes a loop of them: plan, call a tool, read the result, call again, sometimes dozens of calls per task. Failure compounds across the loop. Purely illustrative math: if each call succeeds 95% of the time, a 20-step loop finishes cleanly only about a third of the time (0.95^20 ≈ 0.36). That compounding is why "agents that finish tasks" is the right frame for choosing an inference provider, and why single-turn benchmarks mislead.

Evaluate any LLM API for AI agents on seven criteria:

1. **Function-calling reliability.** Does the model emit schema-valid tool arguments, consistently, at the end of a long context? This is the make-or-break criterion.
2. **Structured output.** JSON mode or grammar-constrained decoding for the non-tool steps (planners, routers, extractors).
3. **Long-context stability.** Agent loops stuff tool outputs into the context. Quality at 32k tokens matters more than quality at 2k.
4. **Latency consistency, not peak speed.** Agents care about the tail (the slowest call in the loop) more than the fastest.
5. **Filter and refusal behavior.** Covered in depth below; it is the most under-tested criterion.
6. **Cost per completed task.** Tokens per run × price × retry multiplier.
7. **Operations.** Rate limits and concurrency that tolerate bursty loops, plus uptime you can build on.

## Which LLM APIs are best for AI agents in 2026?

Seven providers, grouped by category, assessed on the same criteria. We deliberately cite no benchmark numbers here: provider-published figures are self-reported claims, and DeAI's own harnesses have not published results yet. Where pricing matters, check each provider's published pricing page; agent pricing moves monthly.

### OpenRouter: the aggregator

OpenRouter routes to hundreds of models from many underlying hosts behind one OpenAI-compatible API. For agents, the killer feature is fallback routing: if a route errors or refuses, the request can fail over to another host serving the same model, which directly attacks the compounding-failure problem. Tool calling works when the chosen model and route support it, so pin routes in production. Filter and privacy posture vary by underlying provider, so read the per-route terms, not just OpenRouter's. Billing is pay-as-you-go per token.

### Together AI: hosted open-weight catalog

Together AI serves a broad catalog of open-weight models, serverless or on dedicated endpoints, with function calling on tool-capable models such as the Llama 3.3 Instruct family. Dedicated endpoints are the interesting agent option: fixed capacity means more predictable tail latency for long loops. Platform-level moderation applies on hosted endpoints; retention options are described in its terms and privacy pages. Per-token serverless plus dedicated-capacity billing.

### Fireworks AI: hosted open-weight with speed positioning

Fireworks AI markets low-latency serving of open-weight models and supports function calling and structured outputs on tool-capable models; its speed claims are the company's own. For agents, the practical draws are serverless pay-per-token access to popular tool-calling models and on-demand dedicated deployments when you outgrow shared capacity. Platform moderation applies; consult its privacy policy for retention terms.

### Groq: custom-hardware serving

Groq serves open-weight models on its own LPU hardware and claims very high throughput and low latency; again, a vendor claim, not an independent measurement. Where that matters for agents: tight loops feel interactive, and high token throughput shortens long tool-output reads. The model catalog is narrower than the big catalogs, so confirm your tool-calling model is served. Platform moderation applies; per-token tiered pricing on its pricing page.

### DeepInfra: budget-positioned hosting

DeepInfra hosts a wide open-weight catalog and positions itself at the low end of per-token pricing; check its pricing page for current rates. It supports function calling on tool-capable models and is a common choice for cost-sensitive agent workloads where the loop runs thousands of calls a day. Platform moderation applies; retention terms are in its privacy documentation.

### Venice AI: privacy-positioned hosting

Venice AI hosts open-weight models with a privacy-forward pitch: it states that it does not retain user prompts. That is a policy statement, not an independently verified fact, as with every provider's retention claim. Venice also markets a lighter filter posture, which some agent builders seek for sensitive-but-benign domains. API access is OpenAI-compatible, with subscription and per-token options on its pricing page.

### Morpheus: a decentralized inference marketplace

Morpheus is a decentralized inference marketplace: instead of one company's servers, requests route to independent operators who serve open-weight models, and the agent talks to an OpenAI-compatible interface. The design has no single central filter, so behavior, including refusal behavior and logging, depends on the operator serving your request, which you should verify yourself rather than assume. Tool calling depends on the model being served. Pricing is set by market dynamics rather than a single price list.

## How do the providers compare side by side?

| Provider | Type | Tool calling | Filter posture | Stated privacy posture | Billing model |
|---|---|---|---|---|---|
| OpenRouter | Aggregator | Model/route dependent | Varies by underlying provider | Varies by route; check per-provider terms | Per-token credits |
| Together AI | Hosted open-weight | On tool-capable models | Platform moderation | Retention options in its terms | Serverless + dedicated |
| Fireworks AI | Hosted open-weight | On tool-capable models | Platform moderation | See privacy policy | Serverless + dedicated |
| Groq | Hosted, custom hardware | On tool-capable models | Platform moderation | See privacy policy | Per-token tiers |
| DeepInfra | Hosted open-weight | On tool-capable models | Platform moderation | See privacy docs | Per-token, budget-positioned |
| Venice AI | Hosted, privacy-positioned | On tool-capable models | Markets lighter filtering | States no prompt retention (policy claim) | Subscription + per-token |
| Morpheus | Decentralized inference marketplace | Model dependent | No central filter; operator-dependent | No central operator; verify per operator | Market-set pricing |

Filter posture is the real differentiator in this table. The hosted platforms are broadly interchangeable on tool calling and model catalog; where they genuinely diverge is what their moderation layers do to your prompts mid-loop.

## Why do content filters decide whether agents finish tasks?

Moderation layers are tuned for chat, where a refusal costs one turn. In an agent loop, a refusal at step 14 of 20 can fail the whole task. And agents wander into flagged territory constantly: security tooling that generates exploit-adjacent code, OSINT research, medical and legal summarization, creative fiction with dark themes, even log analysis containing attack strings. Worse, refusals are often silent quality failures: the model returns a polite deflection instead of a tool call, and your harness has to detect it.

This cuts both ways, and neutrality matters. If you are building a customer-facing product, a provider's moderation layer may be a feature you want. If you are building internal tooling on benign-but-sensitive prompts, it is a reliability bug. Either way, test your actual prompts against each provider before committing. DeAI's refusal-index methodology scores refusal and redirection behavior across providers and prompt categories; our guide to the [best uncensored AI APIs](/best-uncensored-ai-api) explains the categories and the trade-offs.

## How do you switch providers without rewriting your agent?

Because every provider here speaks the OpenAI schema, the migration is a base-URL and key swap. Keep both in environment variables and you can A/B providers per run:

```python
import os
from openai import OpenAI

client = OpenAI(
    base_url=os.environ["AGENT_API_BASE_URL"],  # provider's /v1 endpoint
    api_key=os.environ["AGENT_API_KEY"],
)

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get current weather for a city",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
    },
}]

resp = client.chat.completions.create(
    model=os.environ.get("AGENT_MODEL", "meta-llama/llama-3.3-70b-instruct"),
    messages=[{"role": "user", "content": "What's the weather in Lisbon? Use the tool."}],
    tools=tools,
    tool_choice="auto",
)
print(resp.choices[0].message)
```

The same call as raw HTTP:

```bash
curl "$AGENT_API_BASE_URL/chat/completions" \
  -H "Authorization: Bearer $AGENT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "meta-llama/llama-3.3-70b-instruct",
    "messages": [{"role": "user", "content": "What is the weather in Lisbon? Use the tool."}],
    "tools": [{"type": "function", "function": {"name": "get_weather", "description": "Get current weather for a city", "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}}}],
    "tool_choice": "auto"
  }'
```

Two harness tips regardless of provider: validate every `tool_call` against your schema and re-ask on parse failure instead of crashing the loop, and log task-completion rate per provider; that metric, not latency, is what you are buying. For picking the underlying model itself, see our [model directory](/models).

## What does agent inference actually cost?

Per-token price lists are the wrong unit. The right unit is cost per completed task: tokens per run (agents burn far more than chat; every tool result re-enters the context) times price, times a retry multiplier for failed calls and refused steps. A provider with a higher sticker price but fewer mid-loop failures can cost less per finished task; a cheap provider that drops tool calls costs more. Open-weight serving across these providers is typically a fraction of frontier-API pricing, but verify on each provider's pricing page and measure completion rates on your own workload.

## FAQ

### What is the best model API for agents?

There is no single winner. The best model API for agents is the one that completes your tool-use loops end to end. Evaluate function-calling reliability, structured output, filter behavior on your own prompts, and cost per finished task, not per-token price.

### What is an AI agent inference provider?

An AI agent inference provider hosts models behind an API, usually OpenAI-compatible, that your agent calls repeatedly during a run. Key traits: reliable tool calling, streaming, concurrency that tolerates loops, and predictable filter behavior.

### What should a function calling API comparison measure?

Schema adherence (valid JSON arguments), multi-tool orchestration across long loops, parallel tool calls, error recovery, and how often safety filters interrupt a benign run. Raw chat benchmarks capture none of these.

### Why do content filters matter for agent workloads?

Agents chain many calls, so one refusal mid-loop can fail the whole task. Providers apply different moderation layers, so test your actual prompts. DeAI's refusal-index methodology scores refusal behavior across providers and prompt categories.

## FAQ

**What is the best model API for agents?**

There is no single winner. The best model API for agents is the one that completes your tool-use loops end to end. Evaluate function-calling reliability, structured output, filter behavior on your own prompts, and cost per finished task — not per-token price.

**What is an AI agent inference provider?**

An AI agent inference provider hosts models behind an API — usually OpenAI-compatible — that your agent calls repeatedly during a run. Key traits: reliable tool calling, streaming, concurrency that tolerates loops, and predictable filter behavior.

**What should a function calling API comparison measure?**

Schema adherence (valid JSON arguments), multi-tool orchestration across long loops, parallel tool calls, error recovery, and how often safety filters interrupt a benign run. Raw chat benchmarks capture none of these.

**Why do content filters matter for agent workloads?**

Agents chain many calls, so one refusal mid-loop can fail the whole task. Providers apply different moderation layers, so test your actual prompts. DeAI's refusal-index methodology scores refusal behavior across providers and prompt categories.

## Sources

- [OpenRouter Documentation](https://openrouter.ai/docs) — OpenRouter
- [Together AI Documentation](https://docs.together.ai) — Together AI
- [Fireworks AI](https://fireworks.ai) — Fireworks AI
- [Groq](https://groq.com) — Groq
- [DeepInfra](https://deepinfra.com) — DeepInfra
- [Venice AI](https://venice.ai) — Venice AI
- [Function Calling Guide](https://platform.openai.com/docs/guides/function-calling) — OpenAI
- [Llama 3.3 70B Instruct Model Card](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct) — Meta (Hugging Face)


---

# The 10 Best Open-Source LLM API Providers in 2026 (Full Comparison)

*By DeAI Newsroom · 2026-08-24 · 9 min read*

Canonical: https://www.deai.org/news/best-open-source-llm-api

> **Summary:** There is no single best open-source LLM API — there are ten credible providers that split into aggregators, serverless generalists, speed specialists, and one decentralized inference marketplace.

The best open-source LLM API in 2026 depends on your workload, but there are 10 credible providers worth shortlisting. This roundup compares hosted open-weight APIs on the same four criteria: model catalog, speed posture, billing model, and data policy. Nearly all speak the OpenAI format, so switching costs are close to zero.

## Key takeaways

- **10 providers, 4 categories:** aggregators, serverless generalists, custom-silicon speed specialists, and one decentralized inference marketplace.
- **All 10 expose an OpenAI-compatible endpoint**, so migrating is typically a 2-line change: base URL plus API key.
- **Per-token serverless billing is the default for 8 of the 10.** Featherless uses a flat subscription and Morpheus settles per request on-chain.
- **Hosted open-weight endpoints typically cost a fraction of frontier-API pricing**, but exact rates change monthly; check each provider's pricing page.
- **Zero providers offer independently verifiable privacy guarantees.** Zero-retention and "operators can't see prompts" claims are policy statements, not technical proof, unless you control the hardware.

## What counts as an open-source LLM API?

Strictly speaking, most "open-source LLMs" are **open-weight**: the weights are publicly downloadable under licenses (Apache 2.0, MIT, or custom ones like Meta's Llama license) that vary in commercial terms. True open source would also publish training data and recipes, which almost no frontier lab does.

An open-source LLM API, then, is a **hosted open-weight model**: a provider runs Llama, Qwen, DeepSeek, Mistral, gpt-oss, or similar weights on its own infrastructure and sells you access over HTTP, usually in the OpenAI-compatible chat-completions format, billed per token. You get open-model portability without owning a single GPU. DeAI's [model catalog](/models) tracks which open-weight releases are actually worth serving.

## How we compared them

This market's head terms are dominated by vendor self-ranking listicles, so a note on method: nobody paid for placement here, and the list is **ordered by category, not ranked**. Every provider is assessed on identical criteria: catalog, speed posture, billing model, and data policy. Where a provider publishes performance or privacy numbers, we label them as vendor-reported claims. The same weights can behave differently behind different providers' filters, so DeAI's refusal-index methodology scores how often hosted endpoints refuse standard benign prompts; the results will be worth consulting once scorecards publish. Positioning described here reflects August 2026; lineups shift monthly.

## The 10 providers

### OpenRouter — best for one key across many backends

OpenRouter is an aggregator: one API key and one OpenAI-compatible endpoint that routes to dozens of upstream hosts. Its catalog is effectively the union of its backends, which makes it the fastest way to compare identical weights across different serving stacks. Billing is per-token pass-through with a fee on top, and data policy is the downstream provider's, so read both sets of terms. Best for prototyping and multi-provider redundancy. If you're weighing life after a single aggregator, the [OpenRouter alternatives](/openrouter-alternatives) rundown maps the exit paths.

### Together AI — best for prototype-to-production on one vendor

Together AI is a serverless generalist with one of the broadest open-weight catalogs (Llama, Qwen, DeepSeek, Mistral, gpt-oss, and a long tail) plus fine-tuning and dedicated endpoints when you outgrow shared capacity. Billing is per-token on serverless with reserved options. The company positions its serving stack as research-grade and publishes throughput numbers on its own site; treat those as vendor-reported.

### Fireworks AI — best for latency-tuned production chat

Fireworks AI competes on serving speed for popular open weights, with a curated-but-broad catalog, function-calling support, and both per-token serverless and dedicated GPU options. Fireworks positions its stack as latency-optimized and publishes its own benchmarks; again, treat those as vendor-reported and validate against your own prompts before committing.

### DeepInfra — best for cost-sensitive production traffic

DeepInfra positions on price: a straightforward serverless API across a wide range of open weights, from flagship sizes down to small specialist models, billed per token. The catalog tends to track whatever is popular on Hugging Face. Best for high-volume or batch workloads where you're comfortable running your own evals.

### Groq — best for latency-critical chat and voice

Groq runs open weights on its own LPU silicon instead of GPUs and advertises high tokens-per-second on Llama-class models. That figure is a vendor claim, but speed-first is the whole product. The catalog is curated: expect the popular models, not the long tail. Billing is per-token, with a free tier for evaluation.

### Cerebras — best for long-generation workloads

Cerebras serves a small set of large open models from its wafer-scale hardware and claims the highest generation throughput available on those models (also vendor-reported). The trade-off is a narrow, curated catalog. Best for workloads built around long outputs like code, reasoning traces, and document drafting, where tokens-per-second shapes the user experience.

### Nebius AI Studio — best for European data-residency requirements

Nebius AI Studio is the serverless API arm of a European GPU cloud. The catalog covers the mainstream open weights and billing is per-token. The differentiator is infrastructure posture: EU-based capacity and data-residency positioning that matters to some compliance teams. As with every provider here, retention terms are a policy document, not a technical guarantee.

### Featherless — best for long-tail model experimentation

Featherless is the billing outlier: a flat monthly subscription for serverless access to a large slice of the Hugging Face catalog, thousands of models rather than dozens. Throughput and context limits vary by plan. Best for eval sweeps, experimentation, and niche models the bigger providers don't bother to host.

### Hugging Face Inference Providers — best for HF-native workflows

Hugging Face's Inference Providers routes requests from a model's hub page to whichever partner backend serves it (several providers on this list are options) under one HF token and one bill. The catalog is whatever is warm on partner infrastructure. Best when the hub is already your source of truth for models.

### Morpheus — best for no single corporate counterparty

Morpheus is a decentralized inference marketplace: instead of one company's data center, an OpenAI-compatible gateway routes your request to independent compute operators, and payments settle on-chain between gateway and operator. The practical consequence is that no single company holds your prompt history, though catalog, uptime, and data handling are operator-level properties, and any privacy claims are policy statements, not independently verified facts. Best for builders who explicitly want no single provider in the trust path.

## Side-by-side at a glance

| Provider | Type | Catalog | Billing | Data-policy posture |
|---|---|---|---|---|
| OpenRouter | Aggregator | Very broad, via backends | Per-token pass-through | Downstream provider's policy applies |
| Together AI | Serverless generalist | Broad, plus fine-tuning | Per-token, dedicated tiers | Published policy; check terms |
| Fireworks AI | Serverless generalist | Broad, speed-tuned | Per-token, dedicated | Published policy; check terms |
| DeepInfra | Serverless generalist | Broad, budget | Per-token | Published policy; check terms |
| Groq | Speed specialist | Curated popular models | Per-token | Published policy; check terms |
| Cerebras | Speed specialist | Curated large models | Per-token | Published policy; check terms |
| Nebius AI Studio | GPU-cloud API | Mainstream, EU-hosted | Per-token | EU data-residency positioning |
| Featherless | Subscription serverless | Very long tail | Flat monthly plans | Published policy; check terms |
| HF Inference Providers | Aggregator | Whatever partners serve | Pass-through via HF | Backend partner's policy applies |
| Morpheus | Decentralized marketplace | Operator-dependent | Per-request, on-chain | Operator-level statements, unverified |

Also worth a look, depending on your niche: SambaNova (custom silicon), Lambda and Hyperbolic (GPU-cloud APIs), Novita AI (budget serverless), Baseten and Replicate (model-infra platforms), and Venice (privacy-positioned).

## Which provider is best for what?

- **Prototyping across many models:** OpenRouter or Featherless, for maximum catalog per unit of integration effort.
- **Production chat at scale:** Together AI, Fireworks, or DeepInfra, roughly in order of how much you value tooling versus price.
- **Latency-critical interfaces:** Groq or Cerebras, accepting the curated catalogs.
- **European data residency:** Nebius.
- **HF-native pipelines:** Hugging Face Inference Providers.
- **No single corporate counterparty:** Morpheus.

Whatever you shortlist, run the same eval prompts against two or three finalists. Identical weights can produce noticeably different outputs across providers because of quantization choices, context-window limits, default system prompts, and content filters.

## How hard is it to switch providers?

Mechanically, it's a two-line change. Every provider on this list exposes an OpenAI-compatible endpoint, so the official SDK works against all of them:

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://api.provider-example.com/v1",  # swap per provider
    api_key="PROVIDER_API_KEY",
)

resp = client.chat.completions.create(
    model="provider-specific-model-id",  # model IDs differ per provider
    messages=[{"role": "user", "content": "Hello"}],
)
print(resp.choices[0].message.content)
```

```bash
curl https://api.provider-example.com/v1/chat/completions \
  -H "Authorization: Bearer PROVIDER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "provider-specific-model-id", "messages": [{"role": "user", "content": "Hello"}]}'
```

The one real migration gotcha is model-ID strings: each provider names the same weights differently, so keep the ID in config, not code.

## What about self-hosting?

If your traffic is steady and predictable, renting GPUs and serving weights yourself with vLLM or SGLang can beat any API on unit cost, at the price of owning capacity planning, scaling, and on-call. APIs win on elasticity and operational burden. Many teams end up hybrid: API for bursts and evaluation, self-hosted for the stable baseline.

## FAQ

### What is the best open model API?

There is no universal best. For one key across many backends, look at aggregators like OpenRouter; for raw generation speed, Groq or Cerebras; for broad catalogs with fine-tuning, Together AI or Fireworks; for no single corporate counterparty, Morpheus. Match the provider to the workload.

### What is a hosted open source LLM?

A model with publicly downloadable weights (Llama, Qwen, DeepSeek, Mistral, gpt-oss) that a provider runs on its own infrastructure and exposes over an API, usually in the OpenAI-compatible format, billed per token.

### Are open-weight model APIs cheaper than frontier APIs?

Usually, yes. Hosted open-weight endpoints typically cost a fraction of frontier-API pricing for comparable capability, though rates change often. Check each provider's published pricing page before committing a workload.

### Is "open source" the same as "open weights"?

Not exactly. Most so-called open-source LLMs are open-weight: the weights are downloadable under licenses that vary in commercial terms. True open source would also publish the training data and full training recipe.

### How hard is it to switch open-weight API providers?

Usually a two-line change: swap the base URL and API key in an OpenAI-compatible client. The real work is re-validating behavior: the same weights can be served with different quantization, context limits, and content filters.

## FAQ

**What is the best open model API?**

There is no universal best. For one key across many backends, look at aggregators like OpenRouter; for raw generation speed, Groq or Cerebras; for broad catalogs with fine-tuning, Together AI or Fireworks; for no single corporate counterparty, Morpheus. Match the provider to the workload.

**What is a hosted open source LLM?**

A model with publicly downloadable weights — Llama, Qwen, DeepSeek, Mistral, gpt-oss — that a provider runs on its own infrastructure and exposes over an API, usually in the OpenAI-compatible format, billed per token.

**Are open-weight model APIs cheaper than frontier APIs?**

Usually, yes. Hosted open-weight endpoints typically cost a fraction of frontier-API pricing for comparable capability, though rates change often. Check each provider's published pricing page before committing a workload.

**Is 'open source' the same as 'open weights'?**

Not exactly. Most so-called open-source LLMs are open-weight: the weights are downloadable under licenses that vary in commercial terms. True open source would also publish the training data and full training recipe.

**How hard is it to switch open-weight API providers?**

Usually a two-line change: swap the base URL and API key in an OpenAI-compatible client. The real work is re-validating behavior — the same weights can be served with different quantization, context limits, and content filters.

## Sources

- [OpenRouter](https://openrouter.ai) — OpenRouter
- [Together AI](https://www.together.ai) — Together AI
- [Fireworks AI](https://fireworks.ai) — Fireworks AI
- [DeepInfra](https://deepinfra.com) — DeepInfra
- [Groq](https://groq.com) — Groq
- [Cerebras](https://www.cerebras.ai) — Cerebras
- [Nebius](https://nebius.com) — Nebius
- [Featherless](https://featherless.ai) — Featherless
- [Hugging Face Inference Providers documentation](https://huggingface.co/docs/inference-providers) — Hugging Face
- [OpenAI Python SDK](https://github.com/openai/openai-python) — OpenAI
- [Llama](https://www.llama.com) — Meta
- [Venice](https://venice.ai) — Venice


---

# The 8 Best Private AI APIs in 2026 (Retention Policies Compared)

*By DeAI Newsroom · 2026-08-24 · 9 min read*

Canonical: https://www.deai.org/news/best-private-ai-api

> **Summary:** All eight providers in this roundup state they do not train on your API prompts by default; they differ on how long they retain data and how you verify that. Rankings reflect documented policy terms as of 2026-08-20 — self-reported claims, not independent audits.

Eight AI APIs make defensible privacy claims in 2026, but they differ sharply on how long they keep your prompts and how you can prove it. This roundup ranks all 8 by their documented retention and training policies as of 2026-08-20, not by benchmarks or marketing. Every privacy claim below is self-reported unless noted.

## Key takeaways

- All 8 ranked providers state they do **not** train on API prompts by default. The real differentiator is retention, not training.
- 3 providers state zero or near-zero retention by default; the other 5 retain data for limited windows under contract.
- 3 privacy models exist: contractual (policy), architectural (confidential computing), and distributed (marketplace routing). DeAI has independently audited 0 of them.
- 2 documents determine your real exposure: the data-usage policy and the subprocessor list.
- 1 config change (base URL plus API key) moves most OpenAI-compatible apps between providers.

## How this ranking works

Every entry below gets the same five-row retention summary, compiled from each provider's published data-usage policy, privacy policy, or trust documentation as of 2026-08-20. Rankings reflect the strength of those documented terms (default retention first, then verifiability), not speed, price, or model quality. Policies change, so treat the linked pages as the source of truth. This policy tracking feeds [DeAI's provider trust scoring](/trust), which monitors terms over time. Where a provider says "we don't store prompts," that is a policy statement, not a verified fact, and it is labeled as such.

## What "private" actually means for an AI API

"Private" bundles four separate questions. **Training:** does the provider use your prompts to improve models? **Retention:** how long are prompts and responses stored, and why? **Access:** who can read the data: employees, subprocessors, or, in a marketplace, individual node operators? **Verifiability:** is the claim backed only by a contract, or by a mechanism like remote attestation you can check yourself? The strongest documented posture is [zero data retention](/zero-data-retention-ai-api), where prompts are processed but never persisted. Even that is a commitment you verify, not a property you can see from the outside.

## The 8 best private AI APIs in 2026

Ordered by documented default retention terms. If you weight contractual enforceability over stated defaults, your order will differ. The rows give you the raw material.

### 1. Tinfoil: confidential computing you can check

Tinfoil serves open-weight models inside hardware confidential-computing enclaves and says prompts are processed without the operator seeing plaintext. What distinguishes its documented posture is remote attestation: clients can cryptographically check which code is running before sending data. That is a verification mechanism rather than a promise, though DeAI has not independently validated the company's claims.

- **Trains on your data:** No, per published policy.
- **Default retention:** None, the company says; processing happens inside an attested enclave.
- **Zero-retention path:** Default posture, per provider.
- **Basis of claim:** Policy statement plus an attestation mechanism designed for client verification.
- **Where to verify:** tinfoil.sh

### 2. Venice: privacy-first by stated default

Venice positions privacy as the product: it says prompts and responses are processed without being stored on its servers, and it serves a catalog of open-weight models behind an OpenAI-compatible API. As with every entry here, the no-storage stance is a policy statement drawn from Venice's own documentation; there is no independent audit to cite.

- **Trains on your data:** No, per published policy.
- **Default retention:** Venice says it does not store prompts or responses.
- **Zero-retention path:** Default posture, per provider.
- **Basis of claim:** Policy statement, self-reported.
- **Where to verify:** venice.ai

### 3. Morpheus: a decentralized inference marketplace

Morpheus is a decentralized inference marketplace: requests route to independent compute operators rather than one company's servers. The project says prompts are not centrally logged. The caveat is structural: retention is ultimately a property of whichever operator serves your request, so the marketplace's stance is a policy statement without a single entity to enforce it. The same scrutiny you'd apply to any aggregator applies here.

- **Trains on your data:** The project says no; enforcement sits at the operator level.
- **Default retention:** No central logging, per project documentation; per-operator behavior may vary.
- **Zero-retention path:** Claimed at the protocol level; not uniformly enforceable across operators.
- **Basis of claim:** Policy statement, self-reported.
- **Where to verify:** the project's public documentation.

### 4. Anthropic: strongest frontier-lab terms

Anthropic's published policy is that API inputs are not used to train models by default, with retention limited to a trust-and-safety window described in its privacy policy. Enterprise customers can negotiate tighter terms. This is a contractual posture from a single accountable entity: a weaker default than the zero-retention specialists above, but straightforward to enforce through a data-processing agreement.

- **Trains on your data:** No, by default, per published policy.
- **Default retention:** A limited trust-and-safety window; see the current policy for the period.
- **Zero-retention path:** Available under certain agreements; ask.
- **Basis of claim:** Contractual commitment from a single provider.
- **Where to verify:** anthropic.com/legal/privacy

### 5. OpenAI: zero retention for eligible organizations

OpenAI states API data is not used for training by default. Prompts are retained for a short abuse-monitoring window under its data-usage policy, and zero data retention is available for eligible customers and endpoints. The terms are thoroughly documented; the catch is that the strongest posture is gated behind eligibility rather than default.

- **Trains on your data:** No, by default, per published policy.
- **Default retention:** A limited abuse-monitoring window; see the current policy for the period.
- **Zero-retention path:** Zero data retention for eligible endpoints and organizations.
- **Basis of claim:** Contractual commitment from a single provider.
- **Where to verify:** openai.com/enterprise-privacy

### 6. Mistral: the EU-jurisdiction option

Mistral's terms for La Plateforme state customer data is not used for training. The differentiator is jurisdiction: as an EU-headquartered provider, GDPR governs its processing, which matters if your threat model includes foreign legal process. Retention specifics live in its terms and privacy policy; read the current versions before committing.

- **Trains on your data:** No, per published terms.
- **Default retention:** Per its terms and privacy policy; check current versions.
- **Zero-retention path:** Not prominently documented; raise it in enterprise discussions.
- **Basis of claim:** Contractual commitment, with EU data-protection law behind it.
- **Where to verify:** mistral.ai

### 7. Together: open-weight hosting with contractual terms

Together hosts a large catalog of open-weight models and states it does not train on customer API data. Retention is standard operational logging under its privacy policy, and tighter arrangements are an enterprise conversation. Documented terms are broadly comparable to other US-based open-weight hosts.

- **Trains on your data:** No, per published policy.
- **Default retention:** Standard operational logging; check the current privacy policy.
- **Zero-retention path:** Not prominently documented; ask.
- **Basis of claim:** Contractual commitment.
- **Where to verify:** together.ai

### 8. Fireworks: open-weight hosting with enterprise controls

Fireworks is another major open-weight inference host and states it does not use customer API data for training. Its documented posture mirrors Together's: contractual non-training commitment, standard operational retention, and enterprise arrangements for stricter handling.

- **Trains on your data:** No, per published terms.
- **Default retention:** Standard operational logging; check the current policy.
- **Zero-retention path:** Enterprise arrangements; ask.
- **Basis of claim:** Contractual commitment.
- **Where to verify:** fireworks.ai

## Which AI APIs don't train on your data?

All eight, by stated default. The retention column is where they diverge (documented terms as of 2026-08-20):

| Provider | Trains on API data (default) | Default retention (as documented) | Zero-retention path | Basis of claim |
|---|---|---|---|---|
| Tinfoil | No | None, provider says | Default posture | Policy + attestation mechanism |
| Venice | No | Not stored, provider says | Default posture | Policy statement |
| Morpheus | Project says no | No central logging, project says; operator-level varies | Protocol-level claim | Policy statement |
| Anthropic | No | Limited trust-and-safety window | Certain agreements | Contract |
| OpenAI | No | Limited abuse-monitoring window | Eligible orgs/endpoints | Contract |
| Mistral | No | Per terms/privacy policy | Ask | Contract (EU law) |
| Together | No | Standard operational logging | Ask | Contract |
| Fireworks | No | Standard operational logging | Enterprise arrangements | Contract |

## What is the most private AI API in 2026?

It depends on your threat model. If the concern is **competitive IP leakage**, any provider on this list with a contractual no-training clause likely suffices. If it is **regulated data**, you need a DPA, a subprocessor list, and possibly EU jurisdiction: Mistral, Anthropic, or an enterprise OpenAI agreement. If it is **the provider itself**, only architectural approaches address it: Tinfoil's attested enclaves, or a decentralized inference marketplace like Morpheus where no single entity holds the logs, with the enforceability caveats noted above.

One discipline applies everywhere: claims like "operators can't see your prompts" are policy statements, not verified facts, until an attestation check or independent audit says otherwise. DeAI tracks how these terms change on the [provider trust page](/trust), and the [zero data retention explainer](/zero-data-retention-ai-api) walks through what the commitment does and does not cover.

## Who didn't make the list

Aggregators such as OpenRouter inherit retention from whichever downstream provider serves the request; they are private only if the weakest link is. Cloud platforms (AWS Bedrock, Azure) carry strong documented terms but a different buying motion than a direct API. And consumer chat apps are a different product from their API siblings, often with different training defaults.

## Switching providers is a one-line change

Most providers here expose an OpenAI-compatible endpoint, so evaluation is cheap:

```python
from openai import OpenAI
import os

client = OpenAI(
    base_url=os.environ["LLM_BASE_URL"],  # provider's OpenAI-compatible endpoint
    api_key=os.environ["LLM_API_KEY"],
)

resp = client.chat.completions.create(
    model=os.environ.get("LLM_MODEL", "your-model-name"),
    messages=[{"role": "user", "content": "Hello"}],
)
print(resp.choices[0].message.content)
```

```bash
curl "$LLM_BASE_URL/chat/completions" \
  -H "Authorization: Bearer $LLM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "your-model-name", "messages": [{"role": "user", "content": "Hello"}]}'
```

Swap the environment variables per provider and run your own prompts against each candidate before signing anything.

## FAQ

### What is a private LLM API?

An inference API whose provider contractually or architecturally limits what happens to your prompts: no training on your data by default, short or zero retention, and a published policy you can point to. "Private" is a policy property, not a marketing label.

### What is the most private AI API in 2026?

On documented terms, providers stating zero retention by default (Tinfoil, Venice, and Morpheus) rank highest, and confidential-computing attestation adds verifiability that pure policy cannot. All such claims remain self-reported; no independent audit exists yet.

### Which AI APIs don't train on your data?

As of 2026-08-20, all eight providers in this roundup (Tinfoil, Venice, Morpheus, Anthropic, OpenAI, Mistral, Together, and Fireworks) state they do not train on API prompts by default. Free tiers and consumer chat apps often operate under different terms, so check each data-usage policy.

### Does zero data retention mean zero risk?

No. Zero retention is a policy commitment, not a proof. It removes stored-data exposure but not in-flight interception or compelled-disclosure risk. Weigh it alongside jurisdiction, subprocessors, and any attestation mechanism, and re-check the policy quarterly, because these terms move.

## FAQ

**What is a private LLM API?**

An inference API whose provider contractually or architecturally limits what happens to your prompts: no training on your data by default, short or zero retention, and a published policy you can point to. 'Private' is a policy property, not a marketing label.

**What is the most private AI API in 2026?**

On documented terms, providers stating zero retention by default (Tinfoil, Venice, Morpheus) rank highest; confidential-computing attestation adds verifiability. All such claims are self-reported — no independent audit exists yet.

**Which AI APIs don't train on your data?**

As of 2026-08-20, the APIs of Tinfoil, Venice, Morpheus, Anthropic, OpenAI, Mistral, Together, and Fireworks all state they don't train on API prompts by default. Free tiers and consumer chat apps often differ — check each data-usage policy.

**Does zero data retention mean zero risk?**

No. Zero retention is a policy commitment, not a proof. It removes stored-data exposure but not in-flight or compelled-disclosure risk. Weigh it alongside jurisdiction, subprocessors, and any attestation mechanism.

## Sources

- [Tinfoil — Confidential AI Inference](https://tinfoil.sh) — Tinfoil
- [Venice AI](https://venice.ai) — Venice
- [Anthropic Privacy Policy](https://www.anthropic.com/legal/privacy) — Anthropic
- [Enterprise Privacy at OpenAI](https://openai.com/enterprise-privacy/) — OpenAI
- [Mistral AI](https://mistral.ai) — Mistral AI
- [Together AI](https://www.together.ai) — Together AI
- [Fireworks AI](https://fireworks.ai) — Fireworks AI


---

# The 7 Best Uncensored AI APIs in 2026 (Refusal Policies Compared)

*By DeAI Newsroom · 2026-08-24 · 8 min read*

Canonical: https://www.deai.org/news/best-uncensored-ai-api

> **Summary:** Seven APIs serve open-weight models with minimal refusals in 2026, and they differ more by architecture than by any verified refusal rate. This roundup ranks them on documented content policies and uncensored model availability — no benchmark claims, because no public.

Seven APIs stand out for serving open-weight models with minimal refusals in 2026: Venice, Morpheus, Featherless, OpenRouter, DeepInfra, Together AI, and Hyperbolic. They are ranked here by documented refusal policies and uncensored model availability, not benchmark scores, because no public refusal-harness data exists yet.

## Key takeaways

- **7 APIs compared on 2 filter layers**: model alignment and platform moderation. The model, not the API, produces most refusals, so "uncensored" mostly means serving low-refusal weights without a second filter on top.
- **0 benchmark scores cited**: DeAI's refusal-index harness has no public data yet, so this ranking rests on documented policies and catalogs, not measured refusal rates.
- **3 checks before integrating**: per-model moderation flags, acceptable-use policy scope, and prompt-retention terms.
- **The top 3 (Venice, Morpheus, Featherless) differ by architecture** (hosted API, marketplace, model catalog), not by verified performance. Nothing public separates them on refusal rate.
- **1 line of code to switch**: every provider here exposes an OpenAI-compatible endpoint, so migration is a base-URL swap.

## What does an "uncensored AI API" actually mean?

Refusals happen at two distinct layers, and conflating them is the most common mistake buyers make.

**Layer 1: the model.** Refusal behavior is trained into weights during alignment (SFT, RLHF, DPO). Frontier APIs serve one aligned model; open-model APIs can serve anything, including fine-tunes deliberately trained for compliance (the Dolphin family is the best-known example) and "abliterated" models, where the refusal direction is surgically removed from the weights. Our explainer on [abliterated and uncensored models](/abliterated-uncensored-models-explained) covers how that works and its trade-offs.

**Layer 2: the platform.** Providers can add input classifiers, output filters, and mandatory system prompts on top of whatever the model would do. Some document this clearly; many don't.

So an "uncensored AI API" is usually a provider that (a) serves low-refusal model variants and (b) adds little or no Layer-2 filtering. DeAI's [refusal-index methodology](/refusals) scores endpoints against a fixed battery of prompts across sensitive categories and records full refusals, partial compliance, and compliance. No public harness data exists yet, so nothing in this article cites measured scores.

One caveat that applies everywhere: every provider below enforces an acceptable-use policy prohibiting illegal content. "Uncensored" describes refusal behavior on lawful-adjacent prompts, not a license for anything.

## How were these ranked?

Ordering reflects two verifiable criteria: the permissiveness of each provider's **documented** platform-level filtering, and the **availability of uncensored or abliterated fine-tunes** in its catalog. Where a provider makes privacy or retention claims, those are labeled as policy statements, not independently verified. Policies change; check current docs before committing.

## The 7 best uncensored AI APIs in 2026

### 1. Venice: the most explicit documented uncensored policy

[Venice](https://venice.ai) is a hosted API built around two public commitments: uncensored model endpoints and no prompt retention. Both are policy statements: Venice says it does not store prompts, but that claim rests on its own documentation, not third-party audit.

- **Filter posture:** documents an uncensored stance; endpoints labeled uncensored serve models without an added platform filter.
- **Model availability:** a curated catalog including uncensored variants; smaller than aggregator catalogs.
- **Watch-outs:** verify which exact weights sit behind each endpoint, and treat retention claims as terms of service, not proof.

### 2. Morpheus: a decentralized inference marketplace

[Morpheus](https://github.com/MorpheusAIs) is a decentralized inference marketplace: independent compute providers serve open-weight models, and consumers connect through OpenAI-compatible endpoints. The decentralization here is a mechanism, not a marketing label; it shapes where filtering can exist.

- **Filter posture:** no single operator imposing a uniform platform filter by design; refusal behavior is whatever the served model does.
- **Model availability:** depends on what providers choose to run; open-weight uncensored fine-tunes are within the marketplace's scope.
- **Watch-outs:** behavior and retention terms vary by provider: you are trusting the operator you route to, and there is no central retention policy to read. That cuts both ways.

### 3. Featherless: the deepest uncensored-model catalog

[Featherless](https://featherless.ai) offers serverless access to a large, rotating catalog of Hugging Face models, which is precisely where uncensored fine-tunes live.

- **Filter posture:** no blanket platform filter documented; filtering is largely model-level, and the catalog includes models trained or abliterated for low refusal.
- **Model availability:** its differentiator: community fine-tunes such as the Dolphin family and abliterated variants appear alongside base models.
- **Watch-outs:** catalog churn is real; confirm the specific model you need is currently served, and read the acceptable-use policy.

### 4. OpenRouter: the widest aggregation, with per-model variation

[OpenRouter](https://openrouter.ai) routes requests to many upstream inference providers through one OpenAI-compatible API, which gives it one of the broadest catalogs available.

- **Filter posture:** varies by upstream provider and model: some endpoints carry moderation, others pass the model's output through. Check per-model documentation before assuming.
- **Model availability:** wide, including uncensored fine-tunes.
- **Watch-outs:** the same model ID can behave differently across upstreams, and retention terms depend on which provider actually serves your request.

### 5. DeepInfra: a broad open-model cloud

[DeepInfra](https://deepinfra.com) hosts a wide range of open-weight models behind an OpenAI-compatible API, typically at a fraction of frontier-API pricing.

- **Filter posture:** no blanket platform filter documented; acceptable-use enforcement applies.
- **Model availability:** strong general catalog; uncensored fine-tunes appear, but verify current availability per model.
- **Watch-outs:** check the published terms for logging and retention specifics.

### 6. Together AI: enterprise-grade open-model hosting

[Together AI](https://www.together.ai) is one of the largest open-model hosts and also sells safety tooling, which tells you its defaults are permissive but guardrails are on the shelf.

- **Filter posture:** hosted chat endpoints generally return the model's own outputs; guard models are optional add-ons. Acceptable-use policy applies.
- **Model availability:** large catalog; some uncensored fine-tunes, so check the current list.
- **Watch-outs:** some models ship with provider-recommended system prompts; know what's prepended before judging refusal behavior.

### 7. Hyperbolic: an open-model cloud worth benchmarking

[Hyperbolic](https://hyperbolic.xyz) hosts popular open weights behind an OpenAI-compatible endpoint and competes on price-performance.

- **Filter posture:** no blanket platform filter documented; acceptable-use enforcement applies.
- **Model availability:** solid mainstream open-model coverage; a thinner selection of explicitly uncensored fine-tunes than catalog-first platforms.
- **Watch-outs:** verify the exact variant served and read current retention terms.

## How do their refusal policies compare?

| Provider | Architecture | Documented platform filter | Uncensored fine-tunes | Retention posture |
|---|---|---|---|---|
| Venice | Hosted API | None on uncensored endpoints, per its docs | Yes, labeled variants | Says no retention (policy statement) |
| Morpheus | Decentralized inference marketplace | None at platform level by design | Depends on providers | Set per provider; no central policy |
| Featherless | Serverless model catalog | No blanket filter documented | Yes, community fine-tunes | Check current terms |
| OpenRouter | Aggregator | Varies by upstream/model | Yes, wide selection | Varies by upstream |
| DeepInfra | Hosted open-model cloud | No blanket filter documented; AUP enforced | Some; check catalog | Check current terms |
| Together AI | Hosted open-model cloud | Optional guard tooling; AUP enforced | Some; check catalog | Check current terms |
| Hyperbolic | Hosted open-model cloud | No blanket filter documented; AUP enforced | Limited; check catalog | Check current terms |

## How do you evaluate a no-refusal AI API yourself?

Don't take any ranking on faith, including this one. Four steps:

1. **Build a probe set.** 30–50 prompts spanning your actual borderline categories, run at temperature 0, with refusals logged. Same prompts, every provider.
2. **Control for the model.** Where two providers serve the same weights, differences you measure are platform-layer; where they don't, you're mostly measuring the model.
3. **Read three documents**: per-model moderation notes, the acceptable-use policy, and the retention/logging terms.
4. **Confirm the weights.** Ask which exact model card sits behind the endpoint.

Switching costs are low because every provider here speaks the OpenAI API. The entire migration is:

```python
from openai import OpenAI
import os

client = OpenAI(
    base_url="https://api.provider-example.com/v1",  # swap per provider
    api_key=os.environ["PROVIDER_API_KEY"],
)

resp = client.chat.completions.create(
    model="provider-model-id",
    messages=[{"role": "user", "content": "Your probe prompt here"}],
)
print(resp.choices[0].message.content)
```

```bash
curl https://api.provider-example.com/v1/chat/completions \
  -H "Authorization: Bearer $PROVIDER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "provider-model-id", "messages": [{"role": "user", "content": "Your probe prompt here"}]}'
```

## What doesn't "uncensored" cover?

Three things. First, **legality**: every provider prohibits illegal content, and jurisdiction-specific rules (CSAM, incitement, regulated advice) apply to you regardless of where inference runs. Second, **downstream responsibility**: if you build a product on a low-refusal endpoint, moderation becomes your engineering problem, so budget for it. Third, **quality**: abliteration and permissive fine-tuning can shift model behavior in ways that don't show up in a refusal count. Evaluate on your workload, not just your probe set.

## FAQ

### What is an uncensored AI API?

An API that serves models with permissive refusal behavior and adds little or no platform-level moderation on top. "Uncensored" describes two layers (the model's alignment tuning and the provider's filters), and acceptable-use policies still apply at every provider.

### Is there an AI API without content filters?

Several providers document minimal platform filtering (Venice, Morpheus, and Featherless among them), but every provider enforces an acceptable-use policy, and the model itself may still refuse some prompts. "No filters" is a spectrum, not a binary.

### Which uncensored LLM API is the best?

There is no verified ranking yet. DeAI's refusal-index methodology scores endpoints on a fixed prompt battery, but no public harness data exists to cite. Until then, compare documented policies, model catalogs, and retention terms, all covered in this guide.

### Are no-refusal AI APIs legal to use?

Yes, in most jurisdictions, for lawful use cases. Every provider prohibits illegal content under its acceptable-use policy, and compliance responsibility shifts to the builder. "Uncensored" changes refusal behavior, not legal obligations.

### Do uncensored APIs log your prompts?

Policies vary. Some providers, such as Venice, say they do not retain prompts. Treat that as a policy statement, not a verified fact. Decentralized marketplaces route prompts to independent operators, so read each provider's terms before sending sensitive data.

## FAQ

**What is an uncensored AI API?**

An API that serves models with permissive refusal behavior and adds little or no platform-level moderation on top. 'Uncensored' describes two layers — the model's alignment tuning and the provider's filters — and acceptable-use policies still apply at every provider.

**Is there an AI API without content filters?**

Several providers document minimal platform filtering — Venice, Morpheus, and Featherless among them — but every provider enforces an acceptable-use policy, and the model itself may still refuse some prompts. 'No filters' is a spectrum, not a binary.

**Which uncensored LLM API is the best?**

There is no verified ranking yet. DeAI's refusal-index methodology scores endpoints on a fixed prompt battery, but no public harness data exists to cite. Until then, compare documented policies, model catalogs, and retention terms — all covered in this guide.

**Are no-refusal AI APIs legal to use?**

Yes, in most jurisdictions, for lawful use cases. Every provider prohibits illegal content under its acceptable-use policy, and compliance responsibility shifts to the builder. 'Uncensored' changes refusal behavior, not legal obligations.

**Do uncensored APIs log your prompts?**

Policies vary. Some providers, such as Venice, say they do not retain prompts — treat that as a policy statement, not a verified fact. Decentralized marketplaces route prompts to independent operators, so read each provider's terms before sending sensitive data.

## Sources

- [Venice AI](https://venice.ai) — Venice AI
- [MorpheusAIs on GitHub](https://github.com/MorpheusAIs) — Morpheus
- [OpenRouter](https://openrouter.ai) — OpenRouter
- [Featherless](https://featherless.ai) — Featherless
- [DeepInfra](https://deepinfra.com) — DeepInfra
- [Together AI](https://www.together.ai) — Together AI
- [Hyperbolic](https://hyperbolic.xyz) — Hyperbolic
- [Uncensor Any LLM with Abliteration](https://huggingface.co/blog/mlabonne/abliteration) — Hugging Face Blog
- [Cognitive Computations (Dolphin models)](https://huggingface.co/cognitivecomputations) — Hugging Face


---

# The Cheapest LLM APIs in 2026 — 14 Providers Priced per Million Tokens

*By DeAI Newsroom · 2026-08-24 · 8 min read*

Canonical: https://www.deai.org/news/cheapest-llm-api

> **Summary:** Fourteen LLM API providers compared on per-million-token pricing, with DeepSeek V4 Flash's $0.14/M input list price (as of 2026-08-20) as the verified low-end anchor. The real answer to 'cheapest' depends on your input/output mix, caching, and batch eligibility.

The cheapest LLM API in 2026 is not one provider. It is a moving target across more than a dozen vendors that reprice monthly. The lowest list price DeAI can verify today: DeepSeek V4 Flash at **$0.14 per million input tokens and $0.28 per million output tokens** (as of 2026-08-20). This is how the 14 providers stack up.

## Key takeaways

- DeepSeek V4 Flash lists at $0.14/M input and $0.28/M output tokens (as of 2026-08-20), the lowest verified price anchor in this 14-provider roundup.
- Every provider here prices per million tokens, but your effective cost swings with input/output mix, prompt caching, context tiers, and batch discounts.
- Open-weight inference specialists (Groq, Together, Fireworks, DeepInfra, Cerebras, Hyperbolic, Nebius) typically price a fraction of frontier-lab flagship rates.
- One aggregator (OpenRouter) and one decentralized inference marketplace (Morpheus) let you route across providers instead of committing to a single price list.
- Switching between OpenAI-compatible providers is a one-line base-URL change, so re-shop prices quarterly, not annually.

## What "cheapest" actually means in 2026

List price per million tokens is the headline number, but your invoice is always `(input tokens × input price) + (output tokens × output price)`. Output tokens are typically priced several times higher than input, so a chatbot that generates long answers and a document pipeline that ingests long contexts have completely different cost profiles on the same API.

A worked example using the one price DeAI can verify: a workload of 1 billion input tokens and 250 million output tokens on DeepSeek V4 Flash costs (1,000 × $0.14) + (250 × $0.28) = $210 at list price. The same traffic mix on a provider with a higher input/output price ratio could cost materially more even if its list input price looks similar.

Four more levers move the real number:

- **Prompt caching.** Repeated system prompts and few-shot prefixes are often billed at a steep discount on cache hits. High cache-hit workloads can land far below list.
- **Batch APIs.** Several providers offer asynchronous batch endpoints at reduced rates for non-latency-sensitive jobs.
- **Context tiers.** Some vendors charge more per token once requests cross a context-length threshold.
- **Quality floor.** The cheapest API is the cheapest *model that passes your evals*. A model that is 40% cheaper but fails 10% of tasks costs more after retries and human review.

## The verified anchor: DeepSeek V4 Flash price

DeepSeek built its market position on aggressive list pricing, and V4 Flash continues that pattern: **$0.14 per million input tokens, $0.28 per million output tokens**, list price, as of 2026-08-20. That is the lowest figure DeAI can independently verify today, which is why it anchors this roundup: not because the rest of the market is necessarily more expensive, but because list prices change weekly and DeAI only states numbers it can source.

Two caveats before you commit volume. First, list price is not effective price: caching, context tiers, and any usage-based discounts live in the fine print of [DeepSeek's API docs](https://api-docs.deepseek.com/). Second, price is only one axis. Throughput limits, regional availability, and data-handling terms matter at production scale. For a hands-on setup walkthrough, see DeAI's guide to [running DeepSeek V4 Flash via API](/run-deepseek-v4-flash-api).

## The 14 cheapest LLM API providers, side by side

Sorted alphabetically. With only one verified price on record, ranking the rest by cost would be guesswork. Check each provider's pricing page for current numbers.

| Provider | Category | Pricing model | Pricing source |
|---|---|---|---|
| Anthropic | Frontier model lab | Per-token tiers by model; batch API | [anthropic.com](https://www.anthropic.com) |
| Cerebras | Fast open-weight inference | Per-token on open-weight models | [cerebras.ai](https://www.cerebras.ai) |
| DeepInfra | Open-weight host | Per-token on popular open-weight models | [deepinfra.com](https://deepinfra.com) |
| DeepSeek | Model lab (direct API) | V4 Flash: $0.14/M in, $0.28/M out (as of 2026-08-20) | [api-docs.deepseek.com](https://api-docs.deepseek.com/) |
| Fireworks AI | Open-weight host | Serverless per-token; dedicated deployments | [fireworks.ai](https://fireworks.ai) |
| Google | Frontier model lab | Gemini API per-token tiers; free dev tier | [ai.google.dev](https://ai.google.dev/pricing) |
| Groq | Fast open-weight inference | Per-token on open-weight models | [groq.com](https://groq.com) |
| Hyperbolic | Open-weight host + GPU cloud | Per-token inference; GPU rental | [hyperbolic.xyz](https://hyperbolic.xyz) |
| Mistral AI | Model lab | Per-token tiers for its own models | [docs.mistral.ai](https://docs.mistral.ai) |
| Morpheus | Decentralized inference marketplace | Market-set rates from independent providers | See [DeAI Price Index](/prices) |
| Nebius | GPU cloud + inference | Per-token inference; GPU capacity | [nebius.com](https://nebius.com) |
| OpenAI | Frontier model lab | Per-token tiers by model; batch API | [openai.com](https://openai.com) |
| OpenRouter | Aggregator | Passes through upstream provider pricing | [openrouter.ai](https://openrouter.ai) |
| Together AI | Open-weight host | Serverless per-token; dedicated endpoints | [together.ai](https://www.together.ai) |

### Frontier labs and model labs

Anthropic, Google, Mistral, OpenAI, and DeepSeek all sell their own models directly. The pattern in 2026: each lab's flagship model carries premium pricing, while its smallest tier competes on price. Google's Gemini API has historically included a free tier useful for evaluation before committing. Anthropic and OpenAI both operate batch APIs for discounted asynchronous work. Buying direct from a model lab removes a middleman, but locks you to that lab's lineup.

### Open-weight inference specialists

Cerebras, DeepInfra, Fireworks, Groq, Hyperbolic, Nebius, and Together compete to serve open-weight models (the Llama, DeepSeek, Qwen, and Mistral families and their peers) at high throughput. Because they serve the same weights, this is the most directly price-comparable segment of the market, and rates typically land at a fraction of frontier-flagship API pricing. Several also rent GPU capacity, so if your volume is steady, compare their per-token serverless pricing against renting dedicated compute and self-serving.

### Aggregators

OpenRouter fronts many upstream providers under one API and one bill, with per-model prices that vary by upstream. It is useful as a comparison surface, since you can see multiple hosts' prices for the same open-weight model in one place, and as a failover layer. Check its fee schedule so you know what the aggregation layer adds.

### Decentralized marketplaces

Morpheus is a decentralized inference marketplace: independent compute providers serve open-weight models, and rates are set by market participants rather than a single vendor's price list. Evaluated on the same criteria as every other row here (current price, throughput, model coverage), marketplace rates are live claims to verify at query time, not a fixed list. DeAI's [Price Index](/prices) is being built to track provider list prices, including marketplace rates, in one place; until it is fully live, the pricing pages linked above are the source of truth.

## How do you compare prices without getting burned?

1. **Normalize to your traffic mix.** Pull a week of real input/output token counts and compute the bill on each candidate's price sheet.
2. **Model your cache hit rate.** If your workload resends long prefixes, caching terms can matter more than list price.
3. **Check batch eligibility.** Offline evaluation, summarization, and embedding jobs are often batch-able at lower effective rates.
4. **Verify the quality floor.** Run your evals against the cheapest candidate before migrating; a cheap model that fails is an expensive model.
5. **Read data-usage terms.** Retention and privacy commitments are policy statements, not verified facts. Evaluate them as contractual terms and weigh them alongside price.
6. **Re-check monthly.** 2026 list prices move fast. A quarterly re-shop cadence is the minimum.

## Switching providers is usually a base-URL swap

Most providers here expose an OpenAI-compatible endpoint, which makes price-shopping operationally cheap:

```python
from openai import OpenAI

client = OpenAI(
    api_key="YOUR_PROVIDER_KEY",
    base_url="https://api.your-provider.example/v1",  # swap this line to switch
)

resp = client.chat.completions.create(
    model="provider-model-name",
    messages=[{"role": "user", "content": "Hello"}],
)
print(resp.choices[0].message.content)
```

```bash
curl https://api.your-provider.example/v1/chat/completions \
  -H "Authorization: Bearer YOUR_PROVIDER_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "provider-model-name", "messages": [{"role": "user", "content": "Hello"}]}'
```

Keep the base URL and model name in config, not code, and a provider migration becomes a deploy-time decision rather than an engineering project.

## FAQ

### What is the cheapest LLM API in 2026?

The lowest list price DeAI can verify today is DeepSeek V4 Flash at $0.14/M input and $0.28/M output tokens (as of 2026-08-20). Cheapest for your workload depends on input/output mix, caching, and batch discounts. Compare current rates on provider pricing pages.

### What is the cheapest AI API for production workloads?

For sustained production traffic, the cheapest AI API is usually the smallest model that passes your evals, served with prompt caching and batch discounts. Open-weight hosts and aggregators typically price well below frontier-API flagship rates; verify current numbers per provider.

### How do I get the lowest cost LLM inference?

Normalize prices to your real input/output token mix, use caching and batch endpoints where offered, route simple tasks to smaller models, and re-check pricing pages monthly. Switching OpenAI-compatible providers is usually a one-line base-URL change.

### What is the DeepSeek V4 Flash price?

DeepSeek lists V4 Flash at $0.14 per million input tokens and $0.28 per million output tokens (as of 2026-08-20). Confirm current rates on DeepSeek's official API docs before committing volume.

## FAQ

**What is the cheapest LLM API in 2026?**

The lowest list price DeAI can verify today is DeepSeek V4 Flash at $0.14/M input and $0.28/M output tokens (as of 2026-08-20). Cheapest for your workload depends on input/output mix, caching, and batch discounts — compare current rates on provider pricing pages.

**What is the cheapest AI API for production workloads?**

For sustained production traffic, the cheapest AI API is usually the smallest model that passes your evals, served with prompt caching and batch discounts. Open-weight hosts and aggregators typically price well below frontier-API flagship rates; verify current numbers per provider.

**How do I get the lowest cost LLM inference?**

Normalize prices to your real input/output token mix, use caching and batch endpoints where offered, route simple tasks to smaller models, and re-check pricing pages monthly. Switching OpenAI-compatible providers is usually a one-line base-URL change.

**What is the DeepSeek V4 Flash price?**

DeepSeek lists V4 Flash at $0.14 per million input tokens and $0.28 per million output tokens (as of 2026-08-20). Confirm current rates on DeepSeek's official API docs before committing volume.

## Sources

- [DeepSeek API Docs](https://api-docs.deepseek.com/) — DeepSeek
- [OpenAI](https://openai.com) — OpenAI
- [Anthropic](https://www.anthropic.com) — Anthropic
- [Google AI for Developers — Gemini API Pricing](https://ai.google.dev/pricing) — Google
- [Mistral AI Documentation](https://docs.mistral.ai) — Mistral AI
- [Groq](https://groq.com) — Groq
- [Together AI](https://www.together.ai) — Together AI
- [Fireworks AI](https://fireworks.ai) — Fireworks AI
- [DeepInfra](https://deepinfra.com) — DeepInfra
- [Cerebras](https://www.cerebras.ai) — Cerebras
- [OpenRouter](https://openrouter.ai) — OpenRouter
- [Hyperbolic](https://hyperbolic.xyz) — Hyperbolic
- [Nebius](https://nebius.com) — Nebius


---

# Chutes (Bittensor) vs Morpheus (2026): Decentralized AI Inference

*By DeAI Newsroom · 2026-08-24 · 8 min read*

Canonical: https://www.deai.org/news/chutes-vs-morpheus

> **Summary:** Chutes and Morpheus both serve open-weight models through OpenAI-compatible APIs, but they decentralize differently: Chutes is a subnet inside Bittensor's incentive network, while Morpheus is a standalone decentralized inference marketplace.

Chutes and Morpheus both serve open-weight models through OpenAI-compatible endpoints, typically at a fraction of frontier-API pricing. The real difference is architecture. Chutes is one subnet inside Bittensor; Morpheus is a standalone decentralized inference marketplace. Below, both are scored on the same 8-criteria rubric, with provider claims labeled.

## Key takeaways

- Chutes and Morpheus are built on 2 distinct decentralization models: a Bittensor subnet (miners + validators, TAO incentives) versus a standalone marketplace of independent compute providers.
- Both expose OpenAI-compatible chat-completions endpoints, so switching from OpenAI, or between the two networks, is a 2-line base-URL change (code below).
- As of 2026-08-20, neither network has published independently verified latency, uptime, or price benchmarks; throughput and usage figures on both sides are self-reported claims.
- Privacy on both networks is a policy statement, not a verified fact: prompts travel to third-party GPU operators unless you verify confidential-computing attestation yourself.
- Both typically price at a fraction of frontier-API rates for the same open-weight models. Check each provider's live pricing page before committing.

## Chutes vs Morpheus at a glance

| Criterion | Chutes | Morpheus |
|---|---|---|
| What it is | Serverless inference subnet (SN64) on Bittensor | Standalone decentralized inference marketplace |
| Supply side | Bittensor miners (GPU operators) rewarded via TAO emissions | Independent compute providers running Morpheus node software |
| Settlement layer | Crypto-native, Bittensor ecosystem | Crypto-native, MOR at the settlement layer |
| API | OpenAI-compatible | OpenAI-compatible |
| Model catalog | Rotating lineup of open-weight models | Rotating lineup of open-weight models |
| Privacy posture | Published claims; not independently verified | Published claims; not independently verified |
| Verified public benchmarks | None as of 2026-08-20 | None as of 2026-08-20 |
| Best fit | Builders in or curious about the Bittensor ecosystem | Builders who want a marketplace outside Bittensor |

## What is Chutes (Bittensor)?

Chutes is a serverless-style inference platform built as a subnet on Bittensor (subnet 64). In Bittensor's design, miners (here, GPU operators) deploy containerized model endpoints, validators score their output quality and availability, and TAO emissions reward the operators that perform. Developers consume the result through a single hosted API that speaks the OpenAI chat-completions schema, with per-token, crypto-native billing.

The catalog comes from the major open-weight families (Llama-, Qwen-, and DeepSeek-class models), and the exact lineup rotates as miners add and remove capacity. Chutes says it processes very large daily token volumes; that figure is self-reported, and DeAI has not independently verified it.

## What is Morpheus?

Morpheus is an open-source project and a decentralized inference marketplace: independent compute providers run the project's node software, routing infrastructure matches incoming requests to available providers, and developers consume the result through an OpenAI-compatible interface. Its settlement layer is crypto-native (the MOR token coordinates provider rewards), but for builders the relevant surface is the API, the model catalog, and the per-request cost.

Morpheus describes its provider network as permissionless and its architecture as minimizing trust in any single operator. As with Chutes' claims, these are project statements rather than independently verified facts. The same standard applies to both networks throughout this article.

## How this comparison works

Both networks are assessed on the same eight criteria: (1) architecture and decentralization, (2) model catalog, (3) API compatibility and migration cost, (4) pricing model, (5) privacy posture, (6) reliability and performance, (7) censorship resistance, and (8) developer experience and ecosystem.

Two ground rules. First, where a number comes from a provider's own dashboard or marketing, it is labeled as a claim. Second, where no verified public data exists (which, as of 2026-08-20, covers most performance and pricing questions on both sides), this article says so instead of guessing. DeAI's [provider tracker](/tracker) lists which decentralized networks currently serve which open-weight models, and our [guide to decentralized AI inference networks](/decentralized-ai-inference-networks) covers the wider field beyond these two.

## Which network is more decentralized?

Both have a permissionless supply side: in principle, anyone with suitable GPUs can earn by serving models on either network. The shapes differ.

Chutes inherits Bittensor's two-sided miner/validator design. Decentralization there depends on how miner rewards and validator scoring power are distributed across the subnet, and on Bittensor's broader governance. Morpheus distributes supply across independent providers running its open-source software; decentralization there depends on how concentrated the provider set and the routing layer are in practice. Neither network publishes the kind of provider-distribution data that would settle the question, so the honest answer is: different architectures, unproven either way.

## Which has the better model catalog?

Both networks serve rotating catalogs of current open-weight models rather than proprietary frontier models. In practice, the popular families overlap heavily, and availability on any given day depends on which operators have capacity online. Neither catalog is verifiably "better." Check the live model list on each network (or DeAI's tracker) for the specific model IDs your workload needs before deciding.

## Are Chutes and Morpheus drop-in OpenAI alternatives?

Functionally, yes. Both expose endpoints that follow the OpenAI chat-completions schema, so migration from a hosted frontier API, or between the two networks, is a base-URL and key swap:

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://<provider-base-url>/v1",  # see each provider's docs
    api_key="<your-api-key>",
)

resp = client.chat.completions.create(
    model="<open-weight-model-id>",
    messages=[{"role": "user", "content": "Summarize decentralized inference in one sentence."}],
)
print(resp.choices[0].message.content)
```

```bash
curl https://<provider-base-url>/v1/chat/completions \
  -H "Authorization: Bearer <your-api-key>" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<open-weight-model-id>",
    "messages": [{"role": "user", "content": "Hello"}]
  }'
```

Check each provider's documentation for the current base URL and exact model IDs. Streaming, tool calling, and structured-output support can vary by model and deployment on both networks, so verify the features you rely on with a small test suite before migrating production traffic.

## Which is cheaper, Chutes or Morpheus?

Neither network publishes independently audited pricing, and this article does not declare a cheaper option. Both meter usage per token and are typically a fraction of frontier-API pricing for the same open-weight models; both settle crypto-natively, which can add conversion and accounting overhead depending on your setup. The only reliable method is to pull each provider's live pricing page, take the three to five models you actually run, and compute your own blended cost per million tokens.

## How private are Chutes and Morpheus, really?

On both networks, your prompt leaves your infrastructure and lands on a third party's GPU. Each project publishes privacy claims about what operators can or cannot see; those are policy statements, not verified facts. No outside party can prove an absence-of-logging claim. If either network offers confidential-computing (TEE) endpoints for specific deployments, treat the attestation as something you verify, not something you assume. For genuinely sensitive workloads, the strongest option on any network remains running the open-weight model on hardware you control.

## Reliability and performance: what can you actually verify?

As of 2026-08-20, neither network has public, independently verified latency or uptime benchmarks. Decentralized supply means heterogeneous hardware, so time-to-first-token and throughput can vary by model, operator, and time of day on both sides. The practical move is identical for each: run your own eval harness on your own prompts against both endpoints for a week, and let your data decide.

## Censorship resistance and refusals

Both networks let independent operators choose which models to serve, which makes unilateral content policy harder to impose than at a single-vendor API. But refusal behavior is primarily a property of the model weights plus any provider-side filtering, not of the network brand. DeAI's refusal-index methodology scores models across sensitive prompt categories; apply that lens to the specific model you're served, whichever network delivers it.

## So, which should you choose?

- **Choose Chutes if** you're already building in or curious about the Bittensor ecosystem and its subnet economics, and its current model lineup covers your needs.
- **Choose Morpheus if** you want a standalone decentralized inference marketplace outside Bittensor, with open-source client and node tooling.
- **Choose either** for cost-sensitive serving of open-weight models. Because both are OpenAI-compatible, you can evaluate both in parallel and switch with a base-URL change. There is no lock-in excuse for not testing.
- **Choose neither, for now,** if you need formal SLAs, compliance paperwork, or verified prompt privacy; a centralized provider or self-hosting will fit those requirements better today.

## FAQ

### What is the difference between Bittensor inference and Morpheus?

Bittensor inference (Chutes is subnet 64) runs inside Bittensor's miner/validator network, with TAO emissions paying GPU operators. Morpheus is a standalone decentralized inference marketplace with its own provider network. Both expose OpenAI-compatible APIs for open-weight models.

### Are Chutes and Morpheus decentralized OpenAI alternatives?

Functionally, yes. Both serve open-weight models through OpenAI-compatible endpoints, so switching is a base-URL change. The trade-offs vs OpenAI: you get open models rather than GPT-series, typically lower per-token prices, and fewer formal SLAs.

### Is Chutes or Morpheus cheaper?

Neither has independently audited pricing as of 2026-08-20. Both are typically a fraction of frontier-API pricing, metered per token. Compare each provider's live pricing page for the exact models you plan to run.

### Do Chutes or Morpheus keep prompts private?

Both route prompts to third-party GPU operators, and each publishes privacy claims. Those are policy statements, not verified facts. Treat prompts as visible to the operator unless you have verified confidential-computing attestation on the specific endpoint.

### Which should I choose: Chutes or Morpheus?

Match the ecosystem: Chutes if you want Bittensor's subnet model; Morpheus if you prefer a standalone marketplace. On identical criteria (models, price, privacy, reliability), neither shows a verified edge today, so test both with your own workload.

## FAQ

**What is the difference between Bittensor inference and Morpheus?**

Bittensor inference (Chutes is subnet 64) runs inside Bittensor's miner/validator network, with TAO emissions paying GPU operators. Morpheus is a standalone decentralized inference marketplace with its own provider network. Both expose OpenAI-compatible APIs for open-weight models.

**Are Chutes and Morpheus decentralized OpenAI alternatives?**

Functionally, yes. Both serve open-weight models through OpenAI-compatible endpoints, so switching is a base-URL change. The trade-offs vs OpenAI: you get open models rather than GPT-series, typically lower per-token prices, and fewer formal SLAs.

**Is Chutes or Morpheus cheaper?**

Neither has independently audited pricing as of 2026-08-20. Both are typically a fraction of frontier-API pricing, metered per token. Compare each provider's live pricing page for the exact models you plan to run.

**Do Chutes or Morpheus keep prompts private?**

Both route prompts to third-party GPU operators, and each publishes privacy claims. Those are policy statements, not verified facts. Treat prompts as visible to the operator unless you have verified confidential-computing attestation on the specific endpoint.

**Which should I choose: Chutes or Morpheus?**

Match the ecosystem: Chutes if you want Bittensor's subnet model; Morpheus if you prefer a standalone marketplace. On identical criteria — models, price, privacy, reliability — neither shows a verified edge today, so test both with your own workload.

## Sources

- [Chutes](https://chutes.ai) — Chutes
- [Bittensor Documentation](https://docs.bittensor.com) — Opentensor Foundation
- [Morpheus](https://github.com/MorpheusAIs/Morpheus) — Morpheus (GitHub)
- [OpenAI API Reference](https://platform.openai.com/docs/api-reference) — OpenAI


---

# What Is Confidential AI Inference? TEEs and Who Offers It (2026)

*By DeAI Newsroom · 2026-08-24 · 10 min read*

Canonical: https://www.deai.org/news/confidential-ai-inference-tee

> **Summary:** Confidential AI inference uses hardware trusted execution environments (TEEs) to keep prompts encrypted while a GPU processes them, and remote attestation turns privacy from a policy promise into a checkable proof.

Confidential AI inference means your prompts and outputs stay encrypted even while a remote GPU processes them: protected by hardware trusted execution environments (TEEs) and verifiable through remote attestation, rather than a privacy policy you have to take on faith. As of August 2026, at least six open-inference providers publicly describe some form of it.

## Key takeaways

- A TEE provides three properties (confidentiality, integrity, and attestation), protecting data **in use**, the one stage that TLS and disk encryption don't cover.
- "Zero retention" is a policy claim; remote attestation is a cryptographic proof. Only one of the two is verifiable AI privacy.
- Protecting GPU inference end-to-end takes two TEEs working together: Intel TDX or AMD SEV-SNP on the CPU, plus NVIDIA's confidential-computing mode on the GPU.
- Six providers (Chutes, Morpheus, NEAR, Oasis, Phala, and Targon) describe TEE-based or TEE-adjacent inference, but how much of it you can independently verify varies widely.
- Attestation proves *which code* is running. Without open-source enclave images and reproducible builds, it can't prove *what that code does* with your prompts.

## What is confidential AI inference?

Standard API inference already encrypts your prompt twice: in transit (TLS) and, usually, at rest on the provider's disks. But at the moment of computation, the model has to see plaintext. Your prompt sits unencrypted in CPU and GPU memory, readable in principle by anyone with host access: the cloud operator, a rogue admin, a compromised hypervisor, or a legal order served on whoever controls the machine.

Confidential AI inference closes that gap. The model and your data run inside a hardware-isolated environment whose memory is encrypted and inaccessible to the host system itself. The operator of the physical machine, the person who could otherwise dump RAM, is locked out by the silicon, not by an employee handbook.

That single shift is what separates the two privacy postures you'll see across the inference market: "trust us" (policies, contracts, audits) versus "verify yourself" (hardware isolation plus cryptographic attestation). Both exist on a spectrum, and most real deployments mix them. But they are not the same product.

## What is TEE inference?

TEE inference is confidential inference implemented with a **trusted execution environment**: a secure region created by the CPU or GPU itself. A TEE gives you three properties:

1. **Confidentiality** — data inside the enclave is encrypted in memory; the host OS, hypervisor, and other tenants cannot read it.
2. **Integrity** — the host cannot tamper with the code or data inside without detection.
3. **Attestation** — the hardware can produce a signed statement proving what code is running inside, which a remote user can verify.

In an AI context, the enclave holds the model weights, the inference stack, and your prompt. The machine's owner can see *that* an enclave exists and roughly how much it computes, but not the contents. This is the mechanism behind most "trusted execution environment AI" offerings in 2026.

### What a TEE does and doesn't protect

TEEs protect data in use. They do not, by themselves, provide:

- **Anonymity.** The provider still sees your IP address and request metadata unless you add network-layer privacy.
- **Immunity to side channels.** Academic work on speculative-execution and cache side channels is years old and ongoing; serious implementations apply mitigations, but "TEE" is not a magic word.
- **Honest application code.** A TEE faithfully runs whatever code the operator loaded, including code that logs your prompt. That's why attestation plus open, reproducible enclave builds matter, as covered below.
- **Availability.** The host can always pull the plug. Confidentiality is not censorship resistance.

## How does remote attestation actually work?

Attestation is the piece that turns a TEE from "the provider says there's an enclave" into something you can check. Simplified, the flow is:

1. When the enclave starts, the hardware computes a **measurement**, a cryptographic hash of the enclave code and configuration.
2. The hardware signs a **quote** containing that measurement, using a key rooted in the chip vendor (Intel, AMD, or NVIDIA).
3. Your client (or a verification service) checks the quote's signature against the vendor's root of trust, confirming a genuine TEE produced it.
4. The enclave binds its attestation to the TLS session, so you know the encrypted channel terminates *inside* the enclave, not on the host.
5. You compare the measurement against the published hash of the provider's open-source enclave build.

Step five is the one most marketing pages skip. Attestation alone proves you're talking to a real enclave running *some* specific code. Only if that code is open source and reproducibly buildable does the measurement tell you anything about what happens to your data inside. "We use TEEs" without published measurements and source is closer to a policy claim than a proof.

## Verifiable AI privacy vs. "trust us"

Most inference providers today offer policy-level privacy: a zero-retention statement, a SOC 2 report, a contract clause. These have value (audits catch sloppy practice, and contracts create liability), but they share a structural weakness: they are statements about the past or present that the user cannot independently check, and absence-claims like "our operators can't see your prompts" are, at the policy layer, exactly that: claims.

Verifiable AI privacy inverts the burden. Instead of asking you to trust that logs are off, the system makes logging technically visible: the enclave measurement would change, and your attestation check would fail. The honest way to read the market is as a spectrum:

- **Policy only** — retention promises, terms of service.
- **Policy + audit** — third-party attestation of controls, point-in-time.
- **TEE, provider-attested** — hardware isolation exists, but verification artifacts aren't exposed to users.
- **TEE, user-verifiable** — attestation quotes, open enclave code, reproducible builds.

Nothing on this list is useless, and TEEs don't eliminate the need for organizational trust entirely (someone still writes and deploys the enclave code). But the further right you go, the less you have to take on faith.

## What is Intel TDX for AI, and what handles the GPU?

Intel TDX (Trust Domain Extensions) creates hardware-isolated virtual machines ("trust domains") on recent Xeon servers, cutting the host and hypervisor out of the trust boundary. AMD's SEV-SNP does the equivalent on EPYC processors, adding integrity protection against a malicious host. For AI workloads, either one protects the CPU side: orchestration, tokenization, networking, and the code that drives the GPU.

The GPU side is its own story. NVIDIA introduced a confidential-computing mode starting with its Hopper-generation data-center GPUs (the H100 class), extending TEE protection to GPU memory and computation, with attestation handled through NVIDIA's own verification service. A serious confidential-inference deployment needs *both* layers (CPU trust domain plus GPU confidential mode) with an encrypted channel between them. A "TEE inference" claim that only covers the CPU leaves the actual model computation exposed, so it's worth asking vendors which half they're describing.

## Who actually offers confidential AI inference in 2026?

The following providers publicly describe TEE-based or TEE-adjacent inference. Inclusion is not endorsement: these are the projects' own descriptions of their systems, and the depth of user-verifiable evidence differs. Listed alphabetically.

### Chutes

Chutes, a serverless open-model inference platform built on Bittensor (subnet 64), says it runs workloads inside TEEs and has published documentation describing its confidential-compute setup. As with every entry here, the practical question for a buyer is whether attestation evidence is exposed to end users at request time or exists only as architecture documentation. Check its docs for the current state.

### Morpheus

Morpheus is a decentralized inference marketplace: rather than operating one fleet, it routes requests to independent providers, so the privacy properties of a given request depend on which provider serves it. The project's materials describe TEE support within its provider ecosystem. With any marketplace model, verification is per-provider and per-endpoint: the attestation checklist below applies to the specific node you land on, not to the network brand.

### NEAR

NEAR's AI arm has described a private-inference offering that runs open-weight models inside TEEs combining Intel TDX with NVIDIA GPU attestation, aimed largely at its agent ecosystem. Its documentation is the place to confirm which models are covered and whether user-side attestation verification is shipped or on the roadmap.

### Oasis

Oasis Protocol, best known for its Sapphire confidential EVM, has extended its confidential-compute stack toward AI through ROFL (Runtime Offchain Logic), a framework for running off-chain workloads, including model inference, inside TEEs with on-chain verifiability. Oasis's approach is infrastructure-oriented: it's a toolkit for builders who want attestation anchored to a public chain rather than a consumer inference endpoint.

### Phala

Phala Network is TEE-native infrastructure: it has operated confidential compute since its earlier SGX-based design and now offers GPU-TEE inference built on Intel TDX and NVIDIA confidential computing. Phala publishes attestation tooling and documentation, which makes it comparatively straightforward to evaluate, though "tooling exists" and "you have personally verified a quote" remain different things.

### Targon

Targon (Bittensor subnet 4) describes itself as a confidential inference network and says its miners run models inside TEEs. Its site and documentation are the authoritative sources for whether attestation quotes are currently user-verifiable and which hardware stack it uses.

## How to evaluate a provider's privacy claims

Whatever the landing page says, five questions cut through it:

1. **Which TEE?** CPU-only (TDX/SEV-SNP) or CPU + GPU (NVIDIA confidential computing)? The answer tells you what fraction of the workload is actually protected.
2. **Is attestation exposed to users?** Can your client request and verify a quote, or is attestation an internal detail?
3. **Is the enclave code open source, with reproducible builds?** Without this, a verified measurement is a hash of a black box.
4. **What is claimed at the policy layer on top?** Retention, logging, and jurisdictional statements still matter for everything the TEE doesn't cover, and they remain claims, not proofs.
5. **What happens to metadata?** Timing, request sizes, and network identity leak outside the enclave by default.

You can browse the providers mentioned here on the [DeAI tracker](/tracker), and the [DeAI trust framework](/trust) explains how DeAI categorizes privacy claims, including why attestation-backed claims are scored in a different category from policy-only statements. DeAI's verification methodology treats "operators can't see prompts" as a policy assertion unless it's backed by measured, open enclave code; the same standard is worth applying to any provider on this list.

Confidential inference is real, shipping, and no longer exotic. But the gap between "we use TEEs" and "you can verify our enclave" is where most of the market still lives. Closing that gap is a purchasing decision, not just a technical one.

## FAQ

### What is TEE inference?

TEE inference runs an AI model inside a trusted execution environment, a hardware-isolated enclave that keeps prompts and outputs encrypted while in use. Remote attestation lets users verify the enclave is genuine and running expected code.

### What is a trusted execution environment for AI?

A TEE is a hardware-protected region of a CPU or GPU (Intel TDX, AMD SEV-SNP, or NVIDIA's confidential-computing mode) that isolates computation so even the machine's operator cannot read the data being processed.

### What is verifiable AI privacy?

Verifiable AI privacy means privacy backed by cryptographic proof (remote attestation of the enclave and its code) rather than policy promises like "zero retention," which are statements users cannot independently check.

### What is Intel TDX for AI?

Intel Trust Domain Extensions (TDX) creates hardware-isolated virtual machines ("trust domains") on Xeon servers. For AI it protects the CPU side of inference; GPU-side protection comes from NVIDIA's confidential-computing mode.

## FAQ

**What is TEE inference?**

TEE inference runs an AI model inside a trusted execution environment — a hardware-isolated enclave that keeps prompts and outputs encrypted while in use. Remote attestation lets users verify the enclave is genuine and running expected code.

**What is a trusted execution environment for AI?**

A TEE is a hardware-protected region of a CPU or GPU — Intel TDX, AMD SEV-SNP, or NVIDIA's confidential-computing mode — that isolates computation so even the machine's operator cannot read the data being processed.

**What is verifiable AI privacy?**

Verifiable AI privacy means privacy backed by cryptographic proof — remote attestation of the enclave and its code — rather than policy promises like 'zero retention,' which are statements users cannot independently check.

**What is Intel TDX for AI?**

Intel Trust Domain Extensions (TDX) creates hardware-isolated virtual machines ('trust domains') on Xeon servers. For AI it protects the CPU side of inference; GPU-side protection comes from NVIDIA's confidential-computing mode.

## Sources

- [Intel Trust Domain Extensions (Intel TDX)](https://www.intel.com/content/www/us/en/developer/tools/trust-domain-extensions/overview.html) — Intel
- [NVIDIA Confidential Computing](https://www.nvidia.com/en-us/data-center/solutions/confidential-computing/) — NVIDIA
- [AMD Secure Encrypted Virtualization (SEV)](https://www.amd.com/en/developer/sev.html) — AMD
- [Phala Network](https://phala.network) — Phala Network
- [Oasis Protocol](https://oasisprotocol.org) — Oasis Protocol Foundation
- [Chutes](https://chutes.ai) — Chutes
- [Targon](https://targon.com) — Targon
- [NEAR](https://near.org) — NEAR


---

# The 6 Decentralized AI Inference Networks That Actually Work in 2026

*By DeAI Newsroom · 2026-08-24 · 8 min read*

Canonical: https://www.deai.org/news/decentralized-ai-inference-networks

> **Summary:** Six decentralized AI inference networks are usable by builders in 2026: Chutes and Targon on Bittensor, Phala, Akash, Morpheus, and newer entrant Darkbloom. Most expose OpenAI-compatible endpoints, so evaluating one is usually a base-URL swap.

Six decentralized AI inference networks are worth a builder's time in 2026: Chutes and Targon (both Bittensor subnets), Phala, Akash, Morpheus, and newer entrant Darkbloom. Most expose OpenAI-compatible endpoints, so trying one is usually a base-URL swap. Here is what each actually offers, and which claims to treat as claims.

## Key takeaways

- Six networks have a usable path to decentralized inference in 2026; two of the six (Chutes and Targon) are subnets on Bittensor, not standalone companies.
- Most expose OpenAI-compatible endpoints, so migrating from a centralized API is a base-URL and API-key change, not a rewrite.
- Privacy assurances split into two tiers: hardware attestation (Phala's TEEs) and policy promises (everyone else). Policy promises, including zero-retention claims, are not independently verified.
- No network on this list publishes independently audited pricing or uptime; treat every number on their sites as a provider claim. Open-weight serving typically costs a fraction of frontier-API pricing, but check each provider's pricing page.
- Akash is the outlier: it rents raw GPUs rather than managed endpoints, so "inference" means running your own stack: maximum control, maximum ops burden.

## What counts as "actually works"?

Most coverage of this category is written for token holders. This list is written for people who ship. Our bar: a developer can plausibly get tokens back today, through documented access, serving open-weight models, without buying hardware. If you want the category primer first, read [what decentralized AI inference is](/what-is-decentralized-ai-inference) and come back.

Five of the six clear that bar on public evidence. Darkbloom is included because builder interest is real and growing. We flag below what we could and could not verify as of late August 2026.

## Is there a Bittensor inference API?

Not a single official one. Bittensor is a network of independent subnets, each a market for some digital commodity; token emissions are the mechanism that pays suppliers to show up. Inference lives on specific subnets, and those subnets are the products builders actually call. The two that matter for LLM serving today are Chutes (subnet 64) and Targon (subnet 4). When people say "Bittensor inference API," they almost always mean one of these two endpoints.

## What is Chutes?

Chutes is a Bittensor subnet (SN64) that hosts open-weight models on compute supplied by the subnet's miners, behind an OpenAI-compatible chat-completions API. It describes itself as serverless: create an account on [chutes.ai](https://chutes.ai), get an API key, pay per token.

The draw is catalog breadth. The popular open-weight families (Llama, DeepSeek, Qwen, and friends) tend to show up quickly as new releases land, though the exact list rotates, so check the live model page before designing around a specific model. The privacy posture is the weak point to reason about: your prompts are processed by third-party miners you do not know. Anything Chutes states about retention or logging is a policy claim, not an audited guarantee. Best fit: fast, cheap access to a wide range of open models for workloads that are not sensitive.

## What is Targon?

Targon is a Bittensor subnet (SN4), built by Manifold Labs, serving open-weight LLMs through an OpenAI-compatible API at [targon.com](https://targon.com). Where Chutes emphasizes catalog breadth, Targon positions on serving performance. Throughput and latency are its headline claims, and like all provider-reported performance figures, they are claims until you benchmark against your own traffic.

The access pattern is the same as Chutes: API key, per-token billing, OpenAI-compatible schema. So is the privacy caveat: requests are served by subnet miners, and retention statements are policy. In practice, many builders try both Bittensor subnets (they are one base-URL apart) and keep whichever behaves better under their workload.

## Can you run AI inference on Akash?

Yes, but Akash AI inference is a different species from everything else here. [Akash](https://akash.network) is a decentralized compute marketplace: you lease GPUs (including data-center cards) from independent providers and deploy containers on them, defined via its Stack Definition Language. There is no managed model API; you run vLLM, SGLang, TGI, or Ollama yourself and expose your own endpoint.

The implications cut both ways. You can serve any open-weight model you can fit in VRAM, with whatever logging, retention, and access controls you configure. Your privacy posture is genuinely yours, though the hardware host still has physical access unless you pursue confidential-computing options. Costs are hourly leases, typically a fraction of hyperscaler GPU pricing. The bill you pay is in ops time: autoscaling, monitoring, and model updates are on you. Best fit: teams with steady traffic who would otherwise rent GPUs from a cloud.

## What is Phala Network's confidential inference?

[Phala](https://phala.network) is the privacy-maximalist entry. It runs workloads inside hardware trusted execution environments (Intel TDX for CPUs and NVIDIA's confidential-computing mode for GPUs), so prompts and weights are encrypted in use, and it publishes attestation reports that let a user verify which code is running inside the enclave.

On paper that is the strongest privacy mechanism on this list, because it replaces "trust our policy" with "verify the hardware." Two honest caveats: the verification burden is on you (most users never check an attestation report), and the details above come from Phala's own documentation. Hardware-rooted claims are stronger than policy claims, but they are still the vendor's description of its system. Access is either a managed, OpenAI-compatible confidential inference API or confidential VMs on Phala Cloud for custom stacks. Expect TEE overhead and a narrower model catalog than the Bittensor subnets. Best fit: sensitive workloads where a policy promise is not enough.

## What is Morpheus?

Morpheus is a decentralized inference marketplace: rather than one company serving you, it routes OpenAI-compatible requests to independent compute providers connected to the network. Model availability depends on which providers are live, so check the current catalog before committing. Access is via API key, with pricing published on its site.

Its privacy framing, that requests are handled by independent operators rather than a single logging entity, is a policy statement: the same category of claim as the Bittensor subnets make, and equally unverified by third parties. Best fit: teams that want marketplace routing across multiple providers behind one familiar endpoint, evaluated on the same criteria as everything else here.

## Does Darkbloom actually work?

Darkbloom is the newest name on this list, and the honest answer is: probably, with homework. As of 2026-08-24, its public footprint is thinner than the other five. We could not verify a detailed model catalog, a full pricing page, or third-party usage reports. We include it because builders keep searching for it, not because we can vouch for it.

If you evaluate Darkbloom, demand the same artifacts you would from any provider here: an OpenAI-compatible endpoint, a written model list, a pricing page, and a data-retention policy in writing. Start with non-sensitive workloads until those exist.

## How do the six networks compare?

| Network | Type | How you call it | Privacy posture (as stated) | Best fit |
|---|---|---|---|---|
| Chutes | Bittensor subnet (SN64) | OpenAI-compatible API | Policy claims; third-party miners process prompts | Broad open-model catalog, fast |
| Targon | Bittensor subnet (SN4) | OpenAI-compatible API | Policy claims | Throughput-oriented LLM serving |
| Phala | TEE compute network | Managed API or confidential VMs | Hardware attestation (verify it yourself) | Sensitive workloads |
| Akash | GPU marketplace | Your own stack on leased GPUs | You control the deployment; host has physical access | Committed capacity, custom stacks |
| Morpheus | Decentralized inference marketplace | OpenAI-compatible API | Policy claims; independent operators serve requests | Marketplace routing behind one endpoint |
| Darkbloom | Newer entrant | Per its docs | Unverified | Early adopters willing to validate |

Read the privacy column as a spectrum: one hardware-verified option, four policy-promise options, one unknown.

## Switching is usually a base-URL swap

For every managed endpoint on this list, migration from a centralized API looks like this:

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://<provider-endpoint>/v1",  # from your provider's docs
    api_key="<your-api-key>",
)

resp = client.chat.completions.create(
    model="<open-weight-model-id>",
    messages=[{"role": "user", "content": "Hello"}],
)
print(resp.choices[0].message.content)
```

```bash
curl https://<provider-endpoint>/v1/chat/completions \
  -H "Authorization: Bearer <your-api-key>" \
  -H "Content-Type: application/json" \
  -d '{"model": "<open-weight-model-id>", "messages": [{"role": "user", "content": "Hello"}]}'
```

That covers Chutes, Targon, Phala's managed API, and Morpheus. On Akash, the endpoint is whichever URL your own deployment exposes.

## How we track these networks

Provider lineups change monthly. Our [tracker](/tracker) monitors which networks are serving which models, and DeAI's refusal-index methodology scores how often providers refuse benign prompts. Results are published there as they complete. Treat this article as the map and the tracker as the live terrain.

## FAQ

**Is there a Bittensor inference API?**
Bittensor itself does not sell a unified inference API. Inference happens on its subnets: builders typically call Chutes (subnet 64) or Targon (subnet 4), both of which expose OpenAI-compatible endpoints for open-weight models.

**What is Chutes?**
Chutes is a Bittensor subnet (SN64) that hosts open-weight models behind an OpenAI-compatible API. It describes itself as serverless: you create an account, get an API key, and pay per token. Check its site for the current model list and pricing.

**What is Targon?**
Targon is a Bittensor subnet (SN4) built by Manifold Labs that serves open-weight LLMs through an OpenAI-compatible API, positioning on throughput and latency. Its performance and privacy statements are provider claims, like any hosted API.

**Can you run AI inference on Akash?**
Yes, but Akash leases raw GPU capacity rather than managed model endpoints. You deploy your own inference stack (vLLM, SGLang, Ollama) on rented GPUs, gaining control over models and logging at the cost of doing your own ops.

**What is Morpheus?**
Morpheus is a decentralized inference marketplace: it routes API requests to independent compute providers through an OpenAI-compatible interface. Its privacy and pricing statements are provider claims, like those of every network on this list.

## FAQ

**Is there a Bittensor inference API?**

Bittensor itself does not sell a unified inference API. Inference happens on its subnets: builders typically call Chutes (subnet 64) or Targon (subnet 4), both of which expose OpenAI-compatible endpoints for open-weight models.

**What is Chutes?**

Chutes is a Bittensor subnet (SN64) that hosts open-weight models behind an OpenAI-compatible API. It describes itself as serverless: you create an account, get an API key, and pay per token. Check its site for the current model list and pricing.

**What is Targon?**

Targon is a Bittensor subnet (SN4) built by Manifold Labs that serves open-weight LLMs through an OpenAI-compatible API, positioning on throughput and latency. Its performance and privacy statements are provider claims, like any hosted API.

**Can you run AI inference on Akash?**

Yes, but Akash leases raw GPU capacity rather than managed model endpoints. You deploy your own inference stack (vLLM, SGLang, Ollama) on rented GPUs, gaining control over models and logging at the cost of doing your own ops.

**What is Morpheus?**

Morpheus is a decentralized inference marketplace: it routes API requests to independent compute providers through an OpenAI-compatible interface. Its privacy and pricing statements are provider claims, like those of every network on this list.

## Sources

- [Bittensor documentation](https://docs.bittensor.com) — Bittensor
- [Chutes](https://chutes.ai) — Chutes
- [Targon](https://targon.com) — Targon (Manifold Labs)
- [Phala Network](https://phala.network) — Phala Network
- [Akash Network](https://akash.network) — Akash Network
- [Morpheus](https://mor.org) — Morpheus
- [OpenAI API reference](https://platform.openai.com/docs/api-reference) — OpenAI


---

# DeepSeek API: Official vs Third-Party Hosts (2026) — Privacy & Price

*By DeAI Newsroom · 2026-08-24 · 8 min read*

Canonical: https://www.deai.org/news/deepseek-api-official-vs-third-party

> **Summary:** DeepSeek's open weights and its official API are two different products: the weights can run anywhere, while the API sends your prompts to DeepSeek-operated servers under PRC jurisdiction.

DeepSeek's official API is the direct route: low list prices, but prompts are processed by DeepSeek under PRC jurisdiction, as its own privacy policy describes. Third-party hosts run the same open weights on US/EU infrastructure under their own retention policies. The model is identical — the decision comes down to one question: whose servers do you trust with your prompts?

## Key takeaways

- This is **two decisions, not one**: which weights, and whose servers run them. DeepSeek's open releases make the second decision swappable without changing the model.
- DeepSeek's published privacy policy states user data is stored on servers in the **People's Republic of China**; several governments restricted the DeepSeek app on official devices in 2025 on that basis.
- Third-party hosts (Fireworks, Together, OpenRouter, others) serve DeepSeek weights from their own infrastructure; their zero-retention language is a **policy statement, not a verified fact**.
- The official API has historically posted the **lowest list prices**; third-party hosts typically price open-weight models at a fraction of frontier closed-API rates. Compare current pricing pages, not blog posts.
- Self-hosting is the **only architecture where no external party handles prompts** — but frontier-scale checkpoints demand multi-GPU capacity.

## Is the DeepSeek API safe?

"Safe" splits into two separate risks that are often conflated.

**Model risk** is low by open-weight standards. DeepSeek's recent flagship checkpoints have shipped publicly (V3 and R1 under permissive licenses), which means the weights have been downloaded, inspected, fine-tuned, and red-teamed by thousands of independent teams. Check the V4 model card on Hugging Face for its exact license terms before commercial use.

**Data-governance risk** is the real question, and it attaches to the *service*, not the model. The official API is operated by DeepSeek, a Hangzhou-based company. Its published privacy policy has stated that collected information is stored on servers located in the People's Republic of China. Read the current version before deciding. Policies do get revised.

So the practical answer: for prototyping, public content, and low-sensitivity workloads, the official API is a reasonable choice on price. For regulated data (health, finance, personal data under GDPR), trade secrets, or anything your compliance team would ask about, the jurisdiction of the operator becomes the deciding factor, and that pushes you toward a third-party host or self-hosting.

## Does DeepSeek send data to China?

With the official app or API, yes. That is what DeepSeek's own privacy policy describes. The concern is jurisdictional: Chinese law can compel companies to cooperate with state authorities, so data held by a PRC-domiciled operator carries a legal exposure that a US- or EU-domiciled operator does not. That is a statement about legal jurisdiction, not evidence of misuse.

The concern was serious enough that, in 2025, bodies including the US Navy, NASA, and the state of Texas restricted DeepSeek's app on official devices, and Italy's data-protection authority moved against it. Those actions targeted the official service.

The key technical point most coverage misses: **open weights contain no telemetry**. A checkpoint is a file of numbers. When Fireworks or Together serves DeepSeek weights, your packets go to their infrastructure, under their retention policy. DeepSeek is not in the loop. "DeepSeek sends data to China" is true of the official service and false of the weights themselves.

## Official API vs third-party hosts: what actually changes?

| Dimension | Official DeepSeek API | Third-party host |
|---|---|---|
| Model weights | DeepSeek's own | Identical open checkpoint |
| Where prompts go | DeepSeek-operated servers, PRC jurisdiction | Host's infrastructure, host's jurisdiction |
| Retention basis | DeepSeek privacy policy | Host's published data policy |
| Price basis | DeepSeek's price sheet | Host's price sheet |
| Feature parity | Reference implementation | Varies: context caps, tool calling, structured output |
| Output consistency | Reference behavior | Can drift if the host serves quantized variants |

Two rows deserve emphasis. First, **feature parity is not automatic**: hosts differ in supported context length, tool-calling behavior, and reasoning controls. Second, **serving configuration matters**: some hosts run quantized versions of large checkpoints to cut serving costs, which can subtly change outputs. If you're migrating a workload, run your own eval prompts against both endpoints before switching.

## Who hosts DeepSeek V4 outside China?

DeepSeek's recent flagships shipped as open weights, and serverless providers have historically listed new DeepSeek checkpoints within days of release. As of 2026-08-20, confirm V4 and V4 Pro availability, context length, and quantization on each provider's model page. The field, on identical criteria:

- **Fireworks AI**: US-based serverless and dedicated inference. Publishes security and data-usage documentation; enterprise tiers carry stricter contractual terms. Pricing on its public pricing page.
- **Morpheus**: a decentralized inference marketplace. Requests are routed to independent compute providers rather than a single operator. Privacy properties depend on the route and the providers' published policies; apply the same checklist you would to any host.
- **OpenRouter**: US-based aggregator. Your data goes to whichever upstream provider serves the request; it publishes per-provider data policies so you can restrict routing. Adds one routing layer to your trust chain.
- **Together AI**: US-based serverless and dedicated endpoints for open-weight models. Publishes data-policy documentation; check retention terms for your tier.
- **Venice**: privacy-positioned provider that says it does not retain user prompts. Treat that as a policy statement from the provider, not an independently verified fact.
- **Self-hosting**: the only route where no third party touches prompts. Frontier-scale MoE checkpoints typically need multi-GPU nodes with large aggregate VRAM, so this is a real infrastructure commitment, not a weekend project.

## What is the cheapest DeepSeek V4 host?

DeAI does not quote live per-token prices here: they change frequently, and we have not published benchmarked price data. What we can give you is the structure of the comparison:

- **Input vs output rates**: output tokens usually cost several times more than input; workloads with long generations skew the bill.
- **Cached input**: the official API has historically offered context-caching and off-peak discounts; several third parties offer prompt caching too. Your cache hit rate can matter more than the list price.
- **Serverless vs dedicated**: dedicated endpoints cost more per unit but buy throughput guarantees.
- **Aggregator routing**: OpenRouter-style routing can arbitrage price across upstreams, at the cost of an extra party in the chain.

As a rule of thumb, the official API has historically been the cheapest list price, with third-party hosts close behind and still typically a fraction of frontier closed-API pricing. Build a small spreadsheet, pull each provider's current pricing page, and model your actual input/output ratio.

## How do you switch hosts without rewriting code?

DeepSeek's official API is OpenAI-compatible, and most third-party hosts expose the same interface. Switching is a base-URL and key swap:

```python
from openai import OpenAI
import os

client = OpenAI(
    api_key=os.environ["PROVIDER_API_KEY"],
    base_url=os.environ["PROVIDER_BASE_URL"],  # your host's OpenAI-compatible endpoint
)

resp = client.chat.completions.create(
    model="deepseek-v4-pro",  # use the exact model ID from your host's model page
    messages=[{"role": "user", "content": "Summarize this quarter's infra spend."}],
)
print(resp.choices[0].message.content)
```

```bash
curl "$PROVIDER_BASE_URL/chat/completions" \
  -H "Authorization: Bearer $PROVIDER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-v4-pro",
    "messages": [{"role": "user", "content": "Summarize this quarters infra spend."}]
  }'
```

The official endpoint is documented at DeepSeek's API docs; each third party publishes its own base URL and model IDs, which often differ from the official naming. For a full walkthrough (keys, model IDs, streaming, and error handling), see our guide to [running DeepSeek V4 Pro via API](/run-deepseek-v4-pro-api).

## How should you read a host's privacy claims?

Every provider in this list publishes privacy language; none of it is self-proving. The checklist:

1. **Operator domicile and regions**: which legal jurisdiction governs the company and the servers?
2. **Retention windows**: how long are prompts and responses kept, and for what stated purpose?
3. **Training use**: are API inputs used for training, and is there an opt-out?
4. **Upstreams and subprocessors**: with aggregators, who actually serves the request?
5. **Contractual tiers**: enterprise zero-retention terms are stronger than marketing pages.
6. **Audits**: SOC 2 or ISO certifications are signals, not proof of a specific claim.

DeAI's provider-trust methodology treats published policies as claims to be tracked, not verified facts. The criteria are documented at [DeAI Trust](/trust).

## FAQ

### Is the DeepSeek API safe?

The weights are open and widely audited; the risk question is the host. DeepSeek's official API processes prompts under PRC jurisdiction per its own privacy policy. Third-party hosts run identical weights under their own retention policies. Match the host to your data sensitivity.

### Does DeepSeek send data to China?

With the official app or API, yes: DeepSeek's published privacy policy states collected data is stored on servers in the PRC. Open weights themselves contain no telemetry. When a US/EU host serves them, your prompts go to that host, not to DeepSeek.

### What is the cheapest DeepSeek V4 host?

Prices change too often to quote reliably. The official API has historically posted the lowest list prices, with third-party hosts close behind. Compare current input/output and cached-token rates on each provider's pricing page, and factor in your cache hit rate.

### What are the best DeepSeek V4 API alternatives?

Third-party serverless hosts (Fireworks, Together), aggregators (OpenRouter), privacy-positioned providers (Venice), decentralized inference marketplaces (Morpheus), or self-hosting the open weights. Pick on jurisdiction, published retention policy, and current per-token price.

## FAQ

**Is the DeepSeek API safe?**

The weights are open and widely audited; the risk question is the host. DeepSeek's official API processes prompts under PRC jurisdiction per its own privacy policy. Third-party hosts run identical weights under their own retention policies — match the host to your data sensitivity.

**Does DeepSeek send data to China?**

With the official app or API, yes: DeepSeek's published privacy policy states collected data is stored on servers in the PRC. Open weights themselves contain no telemetry — when a US/EU host serves them, your prompts go to that host, not to DeepSeek.

**What is the cheapest DeepSeek V4 host?**

Prices change too often to quote reliably. The official API has historically posted the lowest list prices, with third-party hosts close behind. Compare current input/output and cached-token rates on each provider's pricing page, and factor in your cache hit rate.

**What are the best DeepSeek V4 API alternatives?**

Third-party serverless hosts (Fireworks, Together), aggregators (OpenRouter), privacy-positioned providers (Venice), decentralized marketplaces (Morpheus), or self-hosting the open weights. Pick on jurisdiction, published retention policy, and current per-token price.

## Sources

- [DeepSeek API Documentation](https://api-docs.deepseek.com) — DeepSeek
- [DeepSeek — official site and policies](https://www.deepseek.com) — DeepSeek
- [deepseek-ai model cards](https://huggingface.co/deepseek-ai) — Hugging Face
- [Fireworks AI](https://fireworks.ai) — Fireworks AI
- [Together AI](https://www.together.ai) — Together AI
- [OpenRouter](https://openrouter.ai) — OpenRouter
- [Venice](https://venice.ai) — Venice
- [OpenAI Python SDK](https://github.com/openai/openai-python) — OpenAI


---

# Does Anthropic Train on Your Data? What the Policy Actually Says

*By DeAI Newsroom · 2026-08-24 · 9 min read*

Canonical: https://www.deai.org/news/does-anthropic-train-on-your-data

> **Summary:** Anthropic's answer to 'do you train on my data' splits in two: consumer Claude.ai conversations may be used if you've allowed it, while API and business-tier traffic is excluded by default per Anthropic's commercial terms.

Short answer: it depends on which Anthropic product you use. Anthropic says it trains on zero API or business-tier prompts by default, while consumer Claude.ai chats can be used for training only if you've allowed it in your privacy settings. One setting controls the answer, and your retention window.

## Key takeaways

- Anthropic states it uses **zero** API, Claude for Work, and enterprise inputs/outputs for model training by default — a contractual commitment in its commercial terms, not a technical proof.
- Consumer Claude.ai (Free, Pro, Max) conversations are used for training **only if you allow it**; one toggle in Settings → Privacy controls the choice and can be changed at any time.
- Your training choice drives **two different retention windows**: declining keeps a shorter standard window, while allowing extends retention substantially. Check Anthropic's current policy text for exact durations; they have changed before.
- Opting out applies **going forward**; per Anthropic, data already consumed by a training run cannot be removed from trained weights.
- Every "we don't train on your data" statement in the industry is a **policy**, not a mechanism. Workloads that need verifiable isolation should evaluate zero-retention architectures or self-hosted open-weight models.

## Does Anthropic train on Claude.ai chats?

For consumer accounts, the answer is: only with your permission, and the permission is real. When Anthropic updated its consumer terms in 2025, Free, Pro, and Max users were presented with a choice: allow conversations to be used to improve and train Anthropic's models, or decline. That choice lives in the app under Settings → Privacy and can be flipped in either direction later.

A few practitioner-relevant details:

- **The choice is account-level.** It applies to your Claude.ai conversations going forward from when you set it. Anthropic's help center also notes that explicit feedback (rating a response, for example) can separately be used to improve its services, which is standard across chat assistants.
- **The plan, not the tool, determines the rules.** If you use Claude Code through a consumer subscription rather than an API key, check which terms govern your usage. Anthropic's documentation distinguishes subscription tiers from API billing, and the data-use rules attach to the plan. This trips people up: the same model, the same terminal, two different data policies.
- **Trust & safety processing is separate.** Regardless of your training choice, Anthropic says conversations may be processed by automated safety systems, and flagged content can be reviewed and retained under different rules. "Not used for training" does not mean "never read by any system."

So if a colleague pastes a screenshot claiming "Anthropic trains on everything now," the accurate correction is: consumer users were asked, and the answer is in their own settings panel.

## Does Anthropic train on API and enterprise data?

Here Anthropic's position is cleaner. Its commercial terms and Trust Center state that customer content sent through the Anthropic API, Claude for Work, and enterprise arrangements is **not used to train models by default**. That structure (consumer product may train with consent, paid business tier does not) mirrors the broader industry norm for hosted model APIs.

Two caveats worth stating plainly:

1. **This is Anthropic's stated policy.** DeAI has not independently verified it, and neither has anyone else outside Anthropic's auditors. Like every provider's no-training promise, it is a contractual representation you enforce through your agreement, not something you can observe from the outside.
2. **Retention for abuse monitoring still exists.** "We don't train on it" and "we don't store it" are different claims. Anthropic documents retention of API traffic for trust and safety purposes, and offers shorter or zero-retention configurations for qualifying customers. Confirm the availability and terms of those directly with Anthropic, because they have historically depended on the arrangement.

If your workload runs on the API or a business seat, the policy answer to "does Anthropic train on your data" is no by default. Whether a policy answer is sufficient for your data class is a separate question, covered below.

## How long does Anthropic keep your data?

Retention is where the 2025 consumer-terms update actually bit, and it's the detail most explainers skip. Per Anthropic's policy text, the retention window attached to your account depends on your training choice: users who decline training use keep a shorter standard window, while users who allow training use accept a substantially longer one; the policy language has described retention in years for consenting accounts. Deleted chats are removed within a stated period, and content flagged by safety systems can be held longer under separate rules.

Exact day counts are deliberately not quoted here. Anthropic's windows have changed, the numbers differ by tier and by choice, and any figure printed here would be stale the next time the policy page is edited. The [privacy policy](https://www.anthropic.com/legal/privacy) and [consumer terms](https://www.anthropic.com/legal/consumer-tos) are the only sources of truth worth checking.

### Why did the retention window cause a backlash on X?

When the updated terms rolled out, screenshots of the consent prompt circulated widely on X, and the anxiety was specific: the prompt bundled "let us train on your chats" together with the longer retention window, and many users read the screen under deadline pressure as "agree, or something bad happens to your history." Confusion followed about whether the choice applied retroactively to years of old conversations, whether declining meant deletion, and whether the setting even existed for people who dismissed the prompt.

The factual core underneath the virality was narrower: a real choice, a real difference in retention windows, and a real deadline to decide. But the episode was a useful reminder that retention, not training, is often the term with sharper teeth, because it determines how long your prompts exist on someone else's infrastructure at all.

## How do you opt out of Anthropic training?

For consumer accounts, the path is short:

1. Open Claude.ai and go to **Settings**.
2. Find the **Privacy** section.
3. Toggle off the setting that allows your conversations to be used to improve Claude (the label may differ slightly as the UI evolves).
4. If you want old conversations gone, delete them individually or clear history — deletion follows the retention rules in the current policy.

Workspace admins on business plans manage data controls at the organization level, and API customers don't need to do anything: the no-training default is contractual, not a dashboard setting.

One thing opting out cannot do: reach backward. Anthropic, like every model provider, states that data already consumed in a training run cannot be extracted from trained weights. If you allowed training use for a period and then changed your mind, the toggle stops future use; it does not un-train the past. That is not an Anthropic-specific limitation; it is how gradient descent works.

## Is a "we don't train on your data" promise actually a guarantee?

No, and this applies to Anthropic, OpenAI, Google, and every hosted provider on identical footing. A no-training commitment is a **policy statement**: enforceable through contracts and audits, invisible to you as a user. It does not account for subprocessors, compelled disclosure, misconfigured logging, or the simple fact that safety classifiers processed your prompt on the way through even if no trainer ever touched it.

If your threat model requires more than a promise, the options form a spectrum. Self-hosting open-weight models on hardware you control gives you verifiable isolation at the cost of operating the stack. Several hosted providers sell zero-retention endpoints, though "zero retention" is itself usually a policy claim unless paired with confidential-computing attestation. Decentralized inference marketplaces such as Morpheus route requests to independent operators, which changes the trust model rather than eliminating it: you are evaluating operator behavior and protocol design instead of a single vendor's legal page. DeAI's [trust framework](/trust) lays out the criteria (retention terms, jurisdiction, verifiability) for comparing any of these options on the same axes.

## Should you move workloads off Anthropic over this?

For most teams, no. If you're on the API or a business plan, Anthropic's stated default is already no-training, and the correct action is hygiene, not migration: confirm your tier, confirm your settings, delete what you don't need, and read the retention section of the current policy rather than a screenshot of it.

Migration becomes rational when your data class demands verifiable isolation (regulated records, client-confidential code, unreleased research) or when your legal team wants a retention term the standard agreement doesn't offer. Most inference stacks are portable: providers broadly converge on OpenAI-compatible endpoints, so moving a workload is usually a base-URL and key swap plus eval re-validation, not a rewrite. DeAI's [Anthropic migration guide](/migrate-from-anthropic) walks through the mechanics, including how to run open-weight models against the same client code.

Whatever you choose, choose it from the policy text, not from the discourse. The terms will change again; the habit of reading them is the durable control.

## FAQ

### Does Anthropic train on my Claude.ai chats?

Only if you've allowed it. Consumer users (Free, Pro, Max) were asked to choose whether conversations can be used to train models. Check Settings → Privacy in Claude.ai to see and change your current choice.

### Does Anthropic train on API data?

Anthropic's commercial terms state that API and Claude for Work inputs and outputs are not used to train models by default. That is a contractual policy commitment, not a technical proof. Verify the current text on Anthropic's legal pages.

### How long does Anthropic keep my conversations?

It depends on your tier and your training choice. Declining training use keeps a shorter standard retention window; allowing it extends retention substantially. Exact windows have changed before, so read the current privacy policy rather than relying on any summary, including this one.

### Can I opt out after already allowing training use?

Yes. The privacy toggle can be switched off at any time and applies going forward. Per Anthropic, data already consumed in a training run cannot be extracted from trained model weights.

### Is Claude safe for confidential code or client data?

The API and business tiers carry Anthropic's no-training commitment, which satisfies many teams. If you need verifiable isolation rather than a policy promise, consider zero-retention configurations or self-hosting open-weight models, and evaluate providers against the criteria in DeAI's [trust framework](/trust).

## FAQ

**Does Anthropic train on my Claude.ai chats?**

Only if you've allowed it. Consumer users (Free, Pro, Max) were asked to choose whether conversations can be used to train models. Check Settings → Privacy in Claude.ai to see and change your current choice.

**Does Anthropic train on API data?**

Anthropic's commercial terms state that API and Claude for Work inputs and outputs are not used to train models by default. That is a contractual policy commitment, not a technical proof — verify the current text on Anthropic's legal pages.

**How long does Anthropic keep my conversations?**

It depends on your tier and your training choice. Declining training use keeps a shorter standard retention window; allowing it extends retention substantially. Exact windows have changed before — read the current privacy policy.

**Can I opt out after already allowing training use?**

Yes — the privacy toggle can be switched off at any time and applies going forward. Per Anthropic, data already consumed in a training run cannot be extracted from trained model weights.

**Is Claude safe for confidential code or client data?**

The API and business tiers carry Anthropic's no-training commitment, which satisfies many teams. If you need verifiable isolation rather than a policy promise, consider zero-retention configurations or self-hosting open-weight models.

## Sources

- [Anthropic Privacy Policy](https://www.anthropic.com/legal/privacy) — Anthropic
- [Anthropic Consumer Terms of Service](https://www.anthropic.com/legal/consumer-tos) — Anthropic
- [Anthropic Commercial Terms of Service](https://www.anthropic.com/legal/commercial-terms) — Anthropic
- [Anthropic Trust Center](https://trust.anthropic.com) — Anthropic
- [Anthropic Help Center](https://support.anthropic.com) — Anthropic


---

# Does DeepSeek Store Your Data? Jurisdiction and the Policy (2026)

*By DeAI Newsroom · 2026-08-24 · 8 min read*

Canonical: https://www.deai.org/news/does-deepseek-store-your-data

> **Summary:** DeepSeek's own privacy policy says the official app and website store user data — including prompts and uploaded files — on servers in the People's Republic of China, with open-ended retention.

Yes. If you use DeepSeek's official app or website, DeepSeek stores your data, including your prompts, on servers in mainland China, according to the company's own privacy policy. But "DeepSeek" is really three distinct data paths: the consumer app, the developer API, and open weights run elsewhere. Each has a different answer.

## Key takeaways

- DeepSeek's privacy policy states that collected data (prompts, uploaded files, account details) is stored on servers in the People's Republic of China, under 1 consumer policy covering the website and mobile apps.
- The policy's retention language is open-ended: data is kept for as long as necessary for the purposes it was collected, with no fixed deletion window published.
- There are 3 ways to run DeepSeek models: official app, official API, and third-party or self-hosted open weights. Each puts a different entity in charge of your prompts.
- The jurisdiction concern traces to specific statutes: China's 2017 National Intelligence Law (Article 7) obliges organizations to assist state intelligence work, while the 2021 PIPL governs domestic handling of personal data.
- Italy's Garante ordered DeepSeek to stop processing Italian users' data in early 2025, 1 of several government actions. Self-hosting open weights remains the only path where zero prompt data leaves infrastructure you control.

## What does DeepSeek's privacy policy actually say?

The consumer privacy policy covering DeepSeek's website and mobile apps names a Hangzhou-based entity as the data controller and lays out broad collection categories. In plain terms, the policy says DeepSeek collects:

- **Account information:** email or phone number, password, profile details.
- **User content:** the text of your prompts, chat history, and any files you upload.
- **Device and usage data:** IP address, device identifiers, log data, and interaction patterns.
- **Cookies and similar tracking technologies.**

On storage, the policy's data-storage section states that the information collected is kept on secure servers located in the People's Republic of China. On retention, the language is open-ended: data is kept for as long as necessary to fulfill the purposes for which it was collected, rather than for a fixed number of days. On sharing, the policy permits disclosure to affiliated companies, service providers, and authorities where required by law. That is standard-shaped language, but the "required by law" clause is interpreted under Chinese law, which is the crux of the jurisdiction question below.

Two caveats matter for practitioners. First, everything above is a policy statement: it describes what DeepSeek says it does, not what an independent auditor has verified. DeepSeek has not published third-party audits of its logging or retention practices. Second, policies change; read the current text on DeepSeek's site rather than relying on any summary, including this one.

## Why does the China jurisdiction matter?

The concern most security teams raise is not that an engineer in Hangzhou is reading your chats. It is that the data sits within reach of a specific legal system, and that system's rules determine who can compel access and what recourse you have.

Three statutes frame the issue:

- **The 2017 National Intelligence Law.** Article 7 obliges Chinese organizations and citizens to support, assist, and cooperate with national intelligence work. Read broadly, this means a China-based company can be compelled to assist state security agencies and may be barred from disclosing that assistance.
- **The 2021 Personal Information Protection Law (PIPL).** PIPL is a genuine privacy statute that regulates how companies collect and process personal data domestically, but it carves out state security and government functions, so it does not function as a shield against state access the way users sometimes assume.
- **The Cybersecurity Law and related rules**, which impose data-localization and security-review duties on operators in China.

Balance matters here. Most major jurisdictions have lawful-access mechanisms. The US CLOUD Act lets American authorities compel disclosure from US companies even for data stored abroad, and EU member states run their own surveillance regimes. The practical question is not "China bad, elsewhere good" but alignment: which legal system governs the entity holding your prompts, and does that fit your threat model and your compliance obligations? For a developer prototyping a side project, the answer may be "fine." For a company handling EU personal data under GDPR, health data, or client-confidential material, a China-based controller with open-ended retention is usually a hard stop, and regulators have said as much. Italy's Garante ordered DeepSeek to halt processing of Italian users' data in early 2025 after finding the company's responses about its data practices insufficient, and several government bodies in other countries restricted the app on official devices around the same period.

## App, API, or open weights — which "DeepSeek" are you using?

This is where most coverage goes wrong: "DeepSeek" is a model family, a consumer app, and a developer platform, and the data answer differs across all three.

### The official app and website

The consumer chatbot at DeepSeek's website and its mobile apps are governed by the consumer privacy policy described above: prompts, uploads, and account data stored on PRC servers, open-ended retention, China-based controller. If you paste it into the app, assume it is retained in China.

### The official API

The developer platform operates under its own terms, documented on DeepSeek's API docs site, and API data-handling language is typically more limited than the consumer policy. Consumer products tend to reserve broader rights to use content for service improvement. But two things stay the same: the controller is still the China-based entity, and the jurisdiction analysis above still applies. "I use the API, not the app" changes the contract, not the legal system. For a fuller breakdown of the trade-offs, see our guide to the [official DeepSeek API versus third-party providers](/deepseek-api-official-vs-third-party).

### Open weights on third-party infrastructure

DeepSeek releases many of its flagship models as downloadable open weights. Check the model cards on Hugging Face for the current license terms. Once weights are portable, the data question detaches from DeepSeek entirely: whoever runs the inference holds your prompts, and their policy and jurisdiction govern. That host could be a major cloud's managed endpoint, an inference aggregator, your own GPU cluster, or a decentralized inference marketplace such as Morpheus. The evaluation criteria are identical across all of them: retention window, jurisdiction, subprocessors, deletion rights. A decentralized architecture doesn't exempt a provider from those questions; you still need to know what operators can see and log. Self-hosting is the only configuration where the answer is structural rather than contractual: prompts never leave machines you control, because there is no counterparty to make a policy claim about.

## How do you evaluate any AI provider's data policy?

The DeepSeek case is a useful template for reading any provider, domestic or foreign. The checklist that drives DeAI's [provider trust framework](/trust) comes down to seven questions:

1. **Where is data stored, and which entity is the controller?** Geography determines jurisdiction.
2. **What is the retention window?** "As long as necessary" is open-ended; a fixed number of days is enforceable-shaped.
3. **Is user content used for training or service improvement, and can you opt out?**
4. **Who are the subprocessors and affiliates with access?**
5. **Is there a working deletion mechanism, and is deletion verified?**
6. **What legal-access regime applies, and does the provider publish transparency reports?**
7. **Are privacy claims audited?** Zero-retention and "operators can't see prompts" are policy statements unless an independent third party has attested to them.

## What should you do about it?

- **Casual users:** treat the official app like any service that retains everything you type in a jurisdiction you didn't choose. Don't paste sensitive personal, financial, or employer-confidential material into it.
- **Developers:** read the API platform terms separately from the consumer policy, and if data residency matters to your users, run the open weights through a host whose jurisdiction and retention terms you can put in a DPA.
- **Enterprises and regulated teams:** the official app and API will rarely survive a compliance review for regulated data. Self-hosting or a provider with suitable contractual and jurisdictional terms is the realistic path, and the open-weight releases make that path available at all.

## FAQ

### Does DeepSeek store your prompts?

Yes, if you use the official app or website. DeepSeek's privacy policy says it collects user inputs, including prompts and uploaded files, and stores them on servers in the People's Republic of China. That is a policy statement; DeepSeek has not published an independent audit of its logging practices.

### Is DeepSeek's data storage located in China?

According to DeepSeek's consumer privacy policy, the information it collects is stored on secure servers located in the People's Republic of China. The API operates under separate developer terms, so check the platform documentation for the current data-handling language.

### Can I use DeepSeek models without sending data to China?

Yes. DeepSeek releases many models as open weights, so third-party providers and your own hardware can run them. In those cases the host's privacy policy and jurisdiction, not DeepSeek's, govern your prompts. Self-hosting is the only path where data never leaves infrastructure you control.

### Does the DeepSeek API have the same data policy as the app?

Not exactly. The consumer app is governed by DeepSeek's consumer privacy policy, while the API platform has its own developer terms. Both involve a China-based data controller, so the jurisdiction question applies either way. Read both documents before choosing.

### Have any governments restricted DeepSeek over data concerns?

Yes. Italy's data protection authority, the Garante, ordered DeepSeek to stop processing Italian users' data in early 2025, and several government agencies elsewhere restricted the app on official devices, citing data-jurisdiction concerns.

## FAQ

**Does DeepSeek store your prompts?**

Yes, if you use the official app or website. DeepSeek's privacy policy says it collects user inputs — including prompts and uploaded files — and stores them on servers in the People's Republic of China. That is a policy statement; DeepSeek has not published an independent audit of its logging practices.

**Is DeepSeek's data storage located in China?**

According to DeepSeek's consumer privacy policy, the information it collects is stored on secure servers located in the People's Republic of China. The API operates under separate developer terms, so check the platform documentation for the current data-handling language.

**Can I use DeepSeek models without sending data to China?**

Yes. DeepSeek releases many models as open weights, so third-party providers and your own hardware can run them. In those cases the host's privacy policy and jurisdiction — not DeepSeek's — govern your prompts. Self-hosting is the only path where data never leaves infrastructure you control.

**Does the DeepSeek API have the same data policy as the app?**

Not exactly. The consumer app is governed by DeepSeek's consumer privacy policy, while the API platform has its own developer terms. Both involve a China-based data controller, so the jurisdiction question applies either way. Read both documents before choosing.

**Have any governments restricted DeepSeek over data concerns?**

Yes. Italy's data protection authority, the Garante, ordered DeepSeek to stop processing Italian users' data in early 2025, and several government agencies elsewhere restricted the app on official devices, citing data-jurisdiction concerns.

## Sources

- [DeepSeek Privacy Policy](https://www.deepseek.com/) — DeepSeek
- [DeepSeek API Documentation](https://api-docs.deepseek.com/) — DeepSeek
- [deepseek-ai model cards](https://huggingface.co/deepseek-ai) — Hugging Face
- [Translation: Personal Information Protection Law of the People's Republic of China](https://digichina.stanford.edu/) — DigiChina, Stanford University
- [National Intelligence Law of the People's Republic of China (2017)](https://www.chinalawtranslate.com/) — China Law Translate
- [Garante action on DeepSeek's processing of Italian users' data](https://www.garanteprivacy.it/) — Garante per la protezione dei dati personali


---

# Does Google Gemini Train on Your Data? What the Policy Says (2026)

*By DeAI Newsroom · 2026-08-24 · 9 min read*

Canonical: https://www.deai.org/news/does-google-gemini-train-on-your-data

> **Summary:** Google's answer depends on which Gemini you mean. Consumer Gemini apps may use your chats to improve Google products, with human review possible; the paid Gemini API, Vertex AI, and Workspace tiers carry terms saying your data is not used for training.

Yes, on the free consumer Gemini apps, Google says your conversations may be reviewed by people and used to improve its products. No, on the paid Gemini API, Vertex AI, and Workspace tiers, Google's terms say your data is not used for training. One account setting decides which side you're on.

## Key takeaways

- Gemini runs under **2 data regimes**: the consumer apps, where Google says chats may improve its products and be read by human reviewers, and the paid/enterprise tiers, where its terms commit to not training on your data.
- **1 setting**, Gemini Apps Activity in your Google account, controls whether future consumer chats are eligible for product improvement. It is on by default for most accounts.
- **3 surfaces, 3 rulebooks**: the Gemini web/mobile apps, the Gemini API in AI Studio (free tier vs paid tier), and Vertex AI / Gemini for Workspace each carry different terms.
- Turning activity off is not the same as deleting history. Google says snippets already reviewed by humans are stored separately from your account.
- Every "we don't train on your data" sentence, from Google or any provider, is a policy statement, not an independently verified technical property.

## Does Google Gemini train on your data? The short answer

There is no single answer, because "Gemini" is not one product. It is a brand stretched across consumer apps, a developer API, and enterprise cloud services, each with its own contract. The question "does Google Gemini train on your data" only becomes answerable once you know which Gemini you are touching.

For the consumer apps (the chatbot at gemini.google.com and the mobile apps), Google's own documentation says conversations may be used to improve and develop Google products and services, and that human reviewers may read them. That is the "yes" regime, and it is the default.

For the paid developer and enterprise surfaces, Google's published terms say customer prompts and outputs are not used to train its models. That is the "no" regime.

One nuance worth holding onto: in the consumer case, Google does not claim it fine-tunes a model on your specific chat verbatim. It says conversations may be used to "improve and develop" its products, a category that includes training data, evaluation sets, and human annotation. For most people asking this question, that is exactly the exposure they care about.

## Which Gemini are you actually using?

Before changing any settings, figure out which of the three surfaces your data touches:

1. **Consumer Gemini apps.** The web and mobile chatbot, on free or consumer-paid personal accounts. Governed by the [Gemini Apps Privacy Hub](https://support.google.com/gemini/answer/13594961) and the general [Google Privacy Policy](https://policies.google.com/privacy). Paying for a premium consumer subscription does not, by itself, move you onto enterprise data terms; what matters is whether your account is a consumer account or a Workspace/enterprise one.
2. **The Gemini API in Google AI Studio.** This has two tiers with different rules. The [Gemini API Terms of Service](https://ai.google.dev/gemini-api/terms) distinguish the unpaid tier, where Google states prompts may be used to improve its products, from the paid tier, where they are not.
3. **Vertex AI and Gemini for Workspace.** The enterprise surfaces, governed by the [Google Cloud Service Specific Terms](https://cloud.google.com/terms/service-terms) and Workspace's enterprise agreements rather than the consumer privacy hub.

If you are a builder shipping an app, the distinction between surfaces 2 and 3, and between the free and paid API tiers, is where most accidental data exposure happens.

## What does the consumer Gemini policy actually say?

Read the consumer privacy hub closely and four things stand out, all in Google's own words:

- **Human review exists.** Google states that trained reviewers may read and annotate Gemini conversations. It says snippets are disconnected from your Google account before reviewers see them, but the content of the snippet itself can still contain names, code, or anything else you pasted in.
- **Chats feed product improvement.** Google says conversations may be used to improve and develop its products and services, which includes its machine-learning models.
- **Google tells you not to trust it with secrets.** The privacy hub advises users not to enter confidential information or anything they would not want a reviewer to see or Google to use. When a provider's own documentation says that, believe it.
- **Activity saving is the default.** For most adult accounts, Gemini Apps Activity is switched on unless you turn it off.

All of the above are Google's statements about its own practices: policy language, as of August 2026, not third-party verification. Policies also change, so treat the linked pages, not this article, as the source of truth.

## How do you turn off Gemini Apps Activity?

The control lives in your Google account, not in the chat window:

1. Open the Gemini web app and go to Settings, or visit [My Activity](https://myactivity.google.com) and select the Gemini product page.
2. Find **Gemini Apps Activity** and switch the **Keep Activity** control off.
3. On the same page, delete any past activity you do not want stored. Turning the toggle off does not delete history; those are two separate actions.
4. If you manage accounts for a family or team, check each account individually; the setting is per-account.

Three caveats, all from Google's own policy language. First, the toggle is forward-looking: Google says chats after you switch it off are not used for product improvement, but it does not undo what was already collected. Second, Google says it still retains recent conversations for a short window to operate the service and process feedback; the privacy hub states the current duration. Third, conversations that were already reviewed or annotated by human reviewers are stored separately and, per Google, are not removed when you delete your activity.

UI labels shift over time; if a label above does not match what you see, the privacy hub documents the current wording.

## Does the paid Gemini API train on your data?

This is where the answer flips, and where builders need to read tiers carefully.

**Free API tier:** Google's terms state that prompts and responses submitted through the unpaid Gemini API may be used to improve Google products. Practically, you should treat the free API tier like the consumer app: no user data you would not want absorbed into a training pipeline.

**Paid API tier:** the same terms state that Google does not use your prompts or responses for product improvement. If your application handles real user data, this is the tier the terms were written for.

**Vertex AI:** Google's [data governance documentation for generative AI](https://cloud.google.com/vertex-ai/generative-ai/docs/data-governance) and the Cloud Service Specific Terms state that customer data is not used to train Google's foundation models without permission, and wrap that commitment in enterprise machinery: data processing agreements, access controls, audit logging, and regionalization options.

**Gemini for Workspace:** business and education domains are covered by Workspace's enterprise terms rather than the consumer privacy hub. Google states that Workspace prompts are not reviewed by humans or used to improve models outside your organization; admins can verify the current language in [Google Workspace Admin Help](https://support.google.com/a).

The pattern across all three: the moment money and a contract change hands, Google's position moves from "we may use your data" to "we do not." That is not generosity; it is how every major model provider segments consumer data collection from enterprise sales.

## What does a no-training promise actually guarantee?

Less than the marketing implies, more than nothing. A few things worth keeping straight:

- **It is a policy statement, not a proof.** No outsider can verify from the API response whether a provider trained on a prompt. Enterprise tiers add contractual remedies and compliance attestations on top of the promise; a consumer toggle has only the promise.
- **Absence claims are unverifiable by design.** "We don't retain," "we don't train," "operators can't see your prompts," whether from Google, API startups, or anyone else, are claims about the inside of someone else's infrastructure. Treat them as commitments with varying legal backing, not as measured properties.
- **The checklist is portable.** Whatever provider you evaluate, read for the same five things: the retention window, the training-use clause, the human-review clause, subprocessor disclosure, and the deletion mechanics. DeAI tracks these dimensions across providers on the [trust hub](/trust).

## What if you need stronger guarantees than a policy?

If a policy promise is not enough for your workload, the realistic options, on the same criteria:

1. **Self-host open-weight models.** Running a model from one of the major open-weight families on hardware you control means no third party ever sees the prompt. It is the strongest privacy posture available, at the cost of operating the stack yourself.
2. **Enterprise API tiers.** Google's paid tiers and comparable enterprise offerings from other major labs give you a no-training clause with a contract behind it. Suitable for most business data, weaker than self-hosting for truly sensitive material.
3. **Providers advertising zero retention.** Several inference providers publish zero-retention policies. Those are policy statements like any other; weigh them accordingly.
4. **Decentralized inference marketplaces.** Morpheus is one example: prompts route to independent operators rather than a single provider's datacenter. The privacy properties depend on the marketplace's routing and disclosure design, so evaluate them with the same checklist you would apply to any retention policy.

The pragmatic rule: match the tier to the sensitivity of the workload. Consumer chatbots are for content you would not mind a stranger reading. Everything else belongs behind a contract or on your own hardware.

## FAQ

**Does Google Gemini train on my data?**
On the free consumer Gemini apps, Google says yes: chats may be reviewed by people and used to improve its products. On the paid Gemini API, Vertex AI, and Gemini for Workspace, Google's terms say customer data is not used for training.

**How do I stop Gemini from using my chats for training?**
Open Gemini Apps Activity in your Google account and switch off Keep Activity. Google says future chats then are not used for product improvement. Delete past activity separately; human-reviewed snippets may be retained under Google's policy.

**Is the free Gemini API tier private?**
No. Google's Gemini API terms state that prompts submitted on the unpaid tier may be used to improve Google products. The paid tier and Vertex AI carry terms saying your data is not used to train models.

**Does turning off Gemini Apps Activity delete my history?**
No. It applies going forward. You can delete past Gemini activity manually from the same page, but Google says conversations already reviewed by human annotators are stored separately and are not removed by deletion.

**What is the most private way to use AI?**
Run an open-weight model on hardware you control, so no provider ever sees the prompt. Otherwise, compare written retention and training terms across providers; every no-training claim is a policy promise, not independent proof.

## FAQ

**Does Google Gemini train on my data?**

On the free consumer Gemini apps, Google says yes — chats may be reviewed by people and used to improve its products. On the paid Gemini API, Vertex AI, and Gemini for Workspace, Google's terms say customer data is not used for training.

**How do I stop Gemini from using my chats for training?**

Open Gemini Apps Activity in your Google account and switch off Keep Activity. Google says future chats then are not used for product improvement. Delete past activity separately; human-reviewed snippets may be retained under Google's policy.

**Is the free Gemini API tier private?**

No. Google's Gemini API terms state that prompts submitted on the unpaid tier may be used to improve Google products. The paid tier and Vertex AI carry terms saying your data is not used to train models.

**Does turning off Gemini Apps Activity delete my history?**

No. It applies going forward. You can delete past Gemini activity manually from the same page, but Google says conversations already reviewed by human annotators are stored separately and are not removed by deletion.

**What is the most private way to use AI?**

Run an open-weight model on hardware you control — no provider ever sees the prompt. Otherwise, compare written retention and training terms across providers; every no-training claim is a policy promise, not independent proof.

## Sources

- [Gemini Apps Privacy Hub](https://support.google.com/gemini/answer/13594961) — Google
- [Gemini API Terms of Service](https://ai.google.dev/gemini-api/terms) — Google AI for Developers
- [Google Cloud Service Specific Terms](https://cloud.google.com/terms/service-terms) — Google Cloud
- [Data governance and generative AI on Vertex AI](https://cloud.google.com/vertex-ai/generative-ai/docs/data-governance) — Google Cloud
- [Google Privacy Policy](https://policies.google.com/privacy) — Google
- [My Activity](https://myactivity.google.com) — Google
- [Google Workspace Admin Help](https://support.google.com/a) — Google


---

# Does OpenAI Train on Your Data? What the Policy Actually Says (2026)

*By DeAI Newsroom · 2026-08-24 · 9 min read*

Canonical: https://www.deai.org/news/does-openai-train-on-your-data

> **Summary:** OpenAI's answer depends on the product: its policy excludes API and business-tier data from training, while consumer ChatGPT chats may be used unless you flip one Data Controls toggle. These are policy commitments, not independent audits — quoted and dated here as of 2026-08-20.

Yes and no. It depends which OpenAI product you use. As of August 2026, OpenAI's policy states it does not train on API or business-tier data, while consumer ChatGPT conversations may be used to improve models unless you turn off one setting. This is what the policy documents actually say, quoted and dated.

## Key takeaways

- OpenAI's policy splits by product: the API and ChatGPT Business, Enterprise, and Edu are excluded from training by default; consumer Free, Plus, and Pro chats may be used unless you opt out.
- The consumer opt-out is a single toggle (Settings → Data Controls → "Improve the model for everyone"), plus Temporary Chat for one-off sessions.
- Opt-outs are not retroactive: OpenAI's help docs say controls apply going forward, and content already used to train a model is not undone.
- API traffic is still retained for abuse monitoring (historically up to 30 days) even though it is not used for training; Zero Data Retention exists for eligible customers.
- A policy is a promise, not a proof: SOC 2 reports and data processing agreements add assurance, but only architectures where the provider never receives plaintext, like self-hosting open weights, remove reliance on policy entirely.

## Does OpenAI train on your data? The answer splits by product

The most common mistake in this debate is treating "OpenAI" as one product with one policy. It isn't. OpenAI operates at least three distinct tiers, and the training answer differs across them. As of the policy pages reviewed on 2026-08-20:

| Product | Trains on your content by default? | What the policy says |
|---|---|---|
| API (platform.openai.com) | No | Not used to train or improve models unless you explicitly opt in |
| ChatGPT Business / Enterprise / Edu | No | "We do not train on your business data or conversations" |
| ChatGPT Free / Plus / Pro | Possibly | Content "may" be used to improve services, including training, unless you opt out |

So before asking "does OpenAI train on your data," ask which door you're walking through. A developer hitting the completions endpoint and a consumer chatting on the free tier are under materially different terms. Policies also change; everything below is dated, and the linked documents are the source of truth.

## Does OpenAI train on API data?

OpenAI's API data usage policy states that it will not use data submitted by customers via the API "to train or improve our models," with an explicit carve-out for cases where you choose to share data for that purpose (policy language reviewed 2026-08-20). That opt-in clause matters: if you submit feedback through an official program or upload files to create a fine-tuned model, that data is obviously used for the purpose you submitted it for.

Three practical caveats for builders:

1. **"Not used for training" is not "not stored."** OpenAI's platform documentation describes retaining API traffic for abuse and misuse monitoring for a limited window (historically up to 30 days) before deletion. That retention exists even though training is excluded.
2. **Zero Data Retention (ZDR) is a separate arrangement.** For eligible endpoints and organizations, OpenAI offers ZDR, under which request and response bodies are not retained at rest. Eligibility and endpoint coverage are defined in the platform docs and the OpenAI Trust Portal, not assumed.
3. **The commitment is contractual.** The Service Terms and, for enterprise customers, a data processing agreement are what bind OpenAI here. That is a stronger instrument than a marketing page, but it is still a policy commitment rather than a technical property of the system.

For most commercial API workloads, the practical reading is: your prompts and completions are not feeding the next GPT model, but they do pass through, and briefly persist on, OpenAI's infrastructure under an abuse-monitoring regime.

## Does OpenAI train on ChatGPT conversations?

This is where the answer flips for consumers. OpenAI's privacy policy states: "We may use Content you provide us to improve our Services, for example to train the models" (reviewed 2026-08-20). On Free, Plus, and Pro plans, that is the default posture unless you change it.

The controls OpenAI documents:

- **The training toggle.** In Settings → Data Controls, switching off "Improve the model for everyone" excludes your future conversations from training use.
- **Temporary Chat.** A per-conversation mode that OpenAI's help center describes as excluded from training and from your chat history.
- **Business tiers.** For ChatGPT Business, Enterprise, and Edu, OpenAI's enterprise privacy page states: "We do not train on your business data or conversations" (reviewed 2026-08-20). Workspace members don't need to hunt for a toggle; the exclusion is the default.

Two nuances users routinely miss:

**Opting out is not retroactive.** OpenAI's help documentation frames data controls as applying going forward. If your past conversations were used to improve a model while the toggle was on, flipping it does not extract that influence. Deleting a chat removes it from your view and queues it for deletion; it is not a time machine.

**Memory is a separate feature.** ChatGPT's saved-memory and chat-history-reference features personalize responses from your past interactions. That is not model training (your memories are not gradient updates), but it is retained personal context, governed by its own settings. Users who care about training exposure usually care about this too, and conflating the two leads to bad decisions in both directions.

## How do you opt out of ChatGPT training?

On the web or mobile apps, as of August 2026:

1. Open **Settings**.
2. Go to **Data Controls**.
3. Toggle off **"Improve the model for everyone."**
4. For sensitive one-off sessions, start a **Temporary Chat** instead.

If you're on ChatGPT Business, Enterprise, or Edu, OpenAI's policy says no action is needed: training on workspace data is excluded by default. If you're an API developer, the same applies: no toggle exists because none is needed under the API data usage policy.

One habit worth adopting regardless of provider: treat the opt-out as a forward-looking switch and assume anything already sent under the old setting may have been used. If a conversation is truly sensitive, the time to protect it is before you send it.

## Is "we don't train on your data" the same as "no one can see your data"?

No, and this distinction is the one that matters most for practitioners moving real workloads.

A training exclusion addresses one specific use of your data: updating model weights. It does not, by itself, address:

- **Abuse-monitoring retention**, under which content may be stored for a limited window and, in narrow circumstances described in OpenAI's policies, reviewed by authorized personnel or trusted contractors.
- **Legal process**, where providers can be compelled to retain or disclose data.
- **Infrastructure exposure**, since prompts transit and are processed on the provider's systems and subprocessors in plaintext.
- **Feature-level storage** like memory, chat history, and file uploads, each with its own retention behavior.

The honest way to read any provider's privacy page, OpenAI's included, is as a set of policy statements. Some are backed by third-party attestation: OpenAI publishes SOC 2 reports and compliance documentation through its Trust Portal, and enterprise customers can negotiate DPAs and ZDR. Those raise the cost of breaking the promise, but they don't convert the promise into a mathematical property. DeAI maintains running coverage of how providers' stated policies compare on the [provider trust pillar](/trust), and the same standard applies everywhere: a claim is a claim until it's audited, and an audit is a point in time.

## What if a policy isn't enough for your workload?

Match the mechanism to the sensitivity of the data. Roughly in ascending order of assurance:

**Contractual layer.** Enterprise agreements, DPAs, Zero Data Retention, and regional processing commitments. Appropriate for most commercial data; insufficient for data you cannot legally or ethically expose to a third party at all.

**Architectural layer: self-hosting open weights.** Running open-weight models (Llama, Qwen, DeepSeek, Mistral, and peers) on your own hardware or inside your own VPC with tools like Ollama, llama.cpp, or vLLM means no external provider ever receives your prompts. The privacy property comes from the architecture, not from a document. The tradeoff is operational: you own the GPUs, the uptime, and the eval burden.

**Decentralized inference marketplaces.** A decentralized inference marketplace such as Morpheus routes requests to independent operators rather than a single corporate provider. That changes the trust shape: no single entity holds your full history. But operator-side privacy claims there are still policy and protocol statements to evaluate on the same criteria as any provider's, not a free pass.

None of these is universally "best." A team drafting marketing copy has different requirements than a hospital summarizing intake notes. What matters is knowing which layer your workload actually needs, and not mistaking a training opt-out for end-to-end confidentiality.

## FAQ

### Does OpenAI train on ChatGPT conversations?

On consumer plans, OpenAI's policy says it may use chat content to improve models unless you opt out in Data Controls or use Temporary Chat. ChatGPT Business, Enterprise, and Edu are excluded from training by default (OpenAI policy pages, reviewed 2026-08-20).

### Does OpenAI train on API data?

OpenAI's API data usage policy states it does not use data submitted via the API to train or improve models unless you explicitly opt in. API traffic is still retained for abuse monitoring, and Zero Data Retention is available for eligible endpoints (policy reviewed 2026-08-20).

### How do I stop ChatGPT from using my chats for training?

Go to Settings → Data Controls and turn off "Improve the model for everyone," or use Temporary Chat for individual conversations. The change applies going forward; OpenAI's help docs say it does not undo training on content already used.

### Is opting out of OpenAI training retroactive?

No. OpenAI's help documentation states that data controls apply to future conversations. Content already used to improve models is not removed from trained models, and deleting a chat is not the same as reversing training.

### Is OpenAI's privacy policy a guarantee?

It is a contractual commitment, not independent proof. SOC 2 reports, data processing agreements, and Zero Data Retention add assurance. The only way to remove reliance on policy entirely is an architecture where the provider never receives your data, such as self-hosting open-weight models.

## FAQ

**Does OpenAI train on ChatGPT conversations?**

On consumer plans, OpenAI's policy says it may use chat content to improve models unless you opt out in Data Controls or use Temporary Chat. ChatGPT Business, Enterprise, and Edu are excluded from training by default (OpenAI policy pages, reviewed 2026-08-20).

**Does OpenAI train on API data?**

OpenAI's API data usage policy states it does not use data submitted via the API to train or improve models unless you explicitly opt in. API traffic is still retained for abuse monitoring, and Zero Data Retention is available for eligible endpoints (policy reviewed 2026-08-20).

**How do I stop ChatGPT from using my chats for training?**

Go to Settings → Data Controls and turn off 'Improve the model for everyone,' or use Temporary Chat for individual conversations. The change applies going forward; OpenAI's help docs say it does not undo training on content already used.

**Is opting out of OpenAI training retroactive?**

No. OpenAI's help documentation states that data controls apply to future conversations. Content already used to improve models is not removed from trained models, and deleting a chat is not the same as reversing training.

**Is OpenAI's privacy policy a guarantee?**

It is a contractual commitment, not independent proof. SOC 2 reports, data processing agreements, and Zero Data Retention add assurance. The only way to remove reliance on policy entirely is an architecture where the provider never receives your data, such as self-hosting open-weight models.

## Sources

- [OpenAI Privacy Policy](https://openai.com/policies/privacy-policy) — OpenAI
- [OpenAI Enterprise Privacy](https://openai.com/enterprise-privacy) — OpenAI
- [How your data is used to improve model performance](https://help.openai.com/en/articles/5722486) — OpenAI Help Center
- [Data Controls FAQ](https://help.openai.com/en/articles/7730893) — OpenAI Help Center
- [OpenAI Service Terms](https://openai.com/policies/service-terms) — OpenAI
- [Your data — API data usage and retention](https://platform.openai.com/docs/guides/your-data) — OpenAI Platform Documentation
- [OpenAI Trust Portal](https://trust.openai.com) — OpenAI


---

# Does OpenRouter Log Your Prompts? The Router Nuance (2026)

*By DeAI Newsroom · 2026-08-24 · 7 min read*

Canonical: https://www.deai.org/news/does-openrouter-log-your-prompts

> **Summary:** Yes, OpenRouter processes and can retain your prompts — and so can every upstream provider it routes to. The useful question isn't whether one company logs, but how many copies of your prompt exist across the route. Both layers publish policies; neither policy is an audit.

Yes, and that's only half the answer. When you send a prompt through OpenRouter, it crosses two logging layers: OpenRouter's own routers and whichever upstream provider actually serves the model. Each keeps records under its own policy, so the question that matters is how many copies of your prompt exist.

## Key takeaways

- Your prompt crosses **2 logging layers**, the router and the upstream provider, and each applies its own retention policy to its own copy.
- OpenRouter's published privacy policy states that prompts are processed and may be retained at the router level; that is a policy statement, not an audit.
- The **same model** can be served by several providers with **different** data policies. The per-provider labels on each OpenRouter model page are the document that actually matters.
- Privacy settings can restrict routing toward providers that state they don't retain prompts: **1 set of toggles**, but a smaller provider pool to draw from.
- Before sending anything sensitive, run the **3-step check** below: router policy, provider policy, data classification.

## What a router does with your prompt

OpenRouter is not a model provider. It's a router: your request travels from your client to OpenRouter's API edge, and from there to whichever upstream provider serves the model you picked. TLS protects the wire, but it terminates at each hop, so your prompt exists in plaintext inside OpenRouter's infrastructure and again inside the provider's.

The practical consequence: anything either party chooses to log, it can log. "Does OpenRouter log your prompts?" is therefore only half the question. The full question is how many parties handle the prompt and what each one's policy says about its copy.

## Does OpenRouter itself log your prompts?

Per its published [privacy policy](https://openrouter.ai/privacy), OpenRouter processes the content you send and may retain it to operate and secure the service (think abuse prevention, debugging, and legal compliance). So the router-layer answer is yes, logging exists at the router level, and the specifics of what is kept and for how long are defined by the current policy page. Read it directly rather than relying on any summary, including this one, because retention terms change.

### Content logs vs. metadata

Even services that don't retain prompt content almost always retain metadata: timestamps, token counts, the model requested, the provider that served it. Billing makes this unavoidable: a router can't charge you per token without counting tokens. When any service says "we don't log," the follow-up question is "content, or everything?" The answer is usually content-only.

### The API vs. the web chat

Hosted chat interfaces typically persist conversation history by design; that's how the history renders when you come back. That's a different storage decision from API retention, and you shouldn't assume one product's policy covers the other. Check which surface you're actually using.

## The per-provider passthrough nuance

This is the part most people miss. OpenRouter forwards your full prompt (system prompt, retrieved context, user data, all of it) to the upstream provider serving the model. That provider now holds its own copy, governed by its own retention and training policy. Your router-level settings can't rewrite a provider's terms; the provider's policy governs the copy it received.

### Same model, different providers, different policies

A popular open-weight model is often served by several providers at once: a first-party lab's API and multiple third-party hosts running the same weights. Their data policies can differ substantially. First-party providers like OpenAI and Anthropic publish their own data-usage terms ([OpenAI](https://openai.com/policies/privacy-policy), [Anthropic](https://www.anthropic.com/legal/privacy)); third-party hosts publish theirs. OpenRouter surfaces per-provider data-policy information on each [model page](https://openrouter.ai/models). That label, not the model name, is what determines what happens to your prompt.

One more caveat: those labels are self-reported. Nobody has audited the GPU boxes. As DeAI's [trust framework](/trust) puts it, a retention claim is a policy statement unless it's backed by a contract or an independent audit.

### Fallback routing can change your provider mid-session

Unless you pin providers, routing shifts with load, price, and availability. Your Monday request and your Tuesday request for the identical model can land with different providers operating under different policies. For sensitive workloads, accepting default fallback behavior means accepting the loosest policy in the provider pool. Pin explicitly.

## Can OpenRouter or its providers train on your prompts?

Treat this as two separate questions. At the router layer, OpenRouter's policy describes how it may use prompt data, and the account settings include controls over data use. Review them rather than assuming defaults. At the provider layer, each provider's own terms govern whether it trains on the copy it received.

A common industry pattern: paid API tiers typically carry no-training commitments, while free or heavily discounted tiers sometimes reserve more rights. That's a heuristic, not a rule, so read each provider's actual terms. And note the asymmetry: opting out at one layer doesn't bind the other. This is exactly why routing preferences matter.

## What does "zero data retention" routing actually mean?

OpenRouter's privacy controls let you bias or restrict routing toward providers whose stated policy is not to retain prompts. Used well, this meaningfully reduces the number of persistent copies of your data. Three caveats keep it honest:

1. **Smaller pool.** Restricting providers narrows your options, which can affect availability, latency, and price. Check live routing and pricing before committing a workload to it.
2. **Still trust-based.** Zero-retention routing is a policy commitment, not a cryptographic guarantee. There's no remote attestation in a typical router setup; you're relying on the provider's word and its incentive to keep it.
3. **Metadata persists.** Token counts and billing records exist regardless. ZDR shrinks content retention; it doesn't make the route blind.

## How to check before you send: a 3-step routine

1. **Router layer.** Read OpenRouter's current privacy policy and your account's privacy settings. Confirm what the router itself retains and what controls are on.
2. **Provider layer.** Open the specific model page, read the per-provider data-policy labels, and pin providers for sensitive traffic instead of accepting any fallback.
3. **Data classification.** Public or internal content is usually fine at policy-level trust. Secrets, credentials, third-party PII, or regulated data (health, financial) should never rest on policy statements; use contractual zero-retention agreements or self-host. And regardless of logging policy, never paste credentials into any hosted LLM: logs aren't the only exfiltration path.

## When a policy statement isn't strong enough

If your threat model requires more than published promises, the main options are:

- **Enterprise agreements.** First-party providers offer contractual zero-retention and data-processing terms. A signed contract is enforceable in a way a web page isn't.
- **Self-hosting open weights** with serving stacks like vLLM or SGLang. You control the machines, so you control the logs. The cost is operating them.
- **Decentralized inference marketplaces** such as Morpheus, which route to independent operators under a different trust model. Apply the same skepticism: an operator's "we can't see your prompts" is an architecture or policy claim to evaluate, not a verified fact.
- **Other routers and gateways.** The same two-layer analysis applies everywhere: the router sees the prompt, and so does whoever serves it.

If this analysis has you re-evaluating your setup, our guide to [switching from OpenRouter](/switch-from-openrouter) covers the migration mechanics, and the [trust framework](/trust) explains how to weigh any provider's privacy claims.

## FAQ

### Does OpenRouter log your prompts?

Yes. OpenRouter's published privacy policy states that prompts are processed at the router level and may be retained for purposes like abuse prevention. Separately, the upstream provider serving each request keeps its own records under its own policy. Check both layers before sending sensitive data.

### Can the upstream provider see my prompts?

Yes. OpenRouter forwards your full prompt, including system prompts and retrieved context, to whichever provider serves the model. That provider's own retention and training policy governs its copy, and policies differ between providers serving the same model.

### Does OpenRouter train on my data?

OpenRouter's policy describes how it may use prompt data and offers account-level controls; whether an upstream provider trains on data is governed by that provider's own terms. Paid API tiers commonly carry no-training commitments, but treat every such commitment as a policy statement, not proof.

### How do I make OpenRouter usage more private?

Restrict routing to providers that state they don't retain prompts, pin providers for sensitive workloads instead of accepting fallbacks, read the per-provider data-policy labels on each model page, and never paste secrets or third-party PII into any hosted LLM regardless of policy.

### Is a "no logging" policy the same as a guarantee?

No. Retention and zero-training commitments are policy statements by the router and its providers; most are not independently audited. For hard guarantees, use enterprise contracts with zero-retention terms or self-host open-weight models on infrastructure you control.

## FAQ

**Does OpenRouter log your prompts?**

Yes. OpenRouter's published privacy policy states that prompts are processed at the router level and may be retained for purposes like abuse prevention. Separately, the upstream provider serving each request keeps its own records under its own policy. Check both layers before sending sensitive data.

**Can the upstream provider see my prompts?**

Yes. OpenRouter forwards your full prompt — including system prompts and retrieved context — to whichever provider serves the model. That provider's own retention and training policy governs its copy, and policies differ between providers serving the same model.

**Does OpenRouter train on my data?**

OpenRouter's policy describes how it may use prompt data and offers account-level controls; whether an upstream provider trains on data is governed by that provider's own terms. Paid API tiers commonly carry no-training commitments, but treat every such commitment as a policy statement, not proof.

**How do I make OpenRouter usage more private?**

Restrict routing to providers that state they don't retain prompts, pin providers for sensitive workloads instead of accepting fallbacks, read the per-provider data-policy labels on each model page, and never paste secrets or third-party PII into any hosted LLM regardless of policy.

**Is a 'no logging' policy the same as a guarantee?**

No. Retention and zero-training commitments are policy statements by the router and its providers; most are not independently audited. For hard guarantees, use enterprise contracts with zero-retention terms or self-host open-weight models on infrastructure you control.

## Sources

- [Privacy Policy](https://openrouter.ai/privacy) — OpenRouter
- [OpenRouter Documentation](https://openrouter.ai/docs) — OpenRouter
- [OpenRouter Models](https://openrouter.ai/models) — OpenRouter
- [Privacy Policy](https://openai.com/policies/privacy-policy) — OpenAI
- [Privacy Policy](https://www.anthropic.com/legal/privacy) — Anthropic


---

# Does xAI's Grok Train on Your Data? What the Policy Says (2026)

*By DeAI Newsroom · 2026-08-24 · 9 min read*

Canonical: https://www.deai.org/news/does-xai-grok-train-on-your-data

> **Summary:** Under xAI's consumer privacy policy, Grok can train on your conversations and X data by default — opting out is a single settings toggle, and API terms are a separate contract. The policy has churned repeatedly since launch, so every claim here is dated.

Yes. Under xAI's consumer privacy policy, your Grok conversations (and, if you're an X user, your public X data) can be used to train and improve xAI's models by default. One settings toggle is what stands between your chats and the training pipeline. API and enterprise tiers play by separate rules. Here's the policy as of August 2026.

## Key takeaways

- **Default is "train."** On consumer Grok (grok.com, the mobile apps, and inside X), xAI's privacy policy has allowed user interactions to be used for training unless you opt out, and one toggle controls it (as of 2026-08-20).
- **Your X posts are in scope.** Public X data has been used to train Grok; X says private accounts and direct messages are excluded: a policy statement, not an independent audit.
- **The API is a different contract.** xAI's API and enterprise terms are separate documents from the consumer privacy policy. Never assume one covers the other.
- **This policy churns.** It has been revised repeatedly since Grok's launch, and regulators have forced changes before. Re-verify before you rely on anything here.
- **Zero exposure requires zero third parties.** The only setup with no provider-side data question is running open weights yourself; everything else is a policy promise.

## Does Grok train on your data? The short answer

For the consumer product, the answer has consistently been "yes, unless you opt out." xAI's privacy policy has stated that the company may use user interactions with Grok (prompts, outputs, and feedback) to train and improve its models, and that X platform data flows into the same pipeline. That posture was still in place as of 2026-08-20, but this policy gets revised often, so treat any summary (including this one) as a snapshot.

Two important carve-outs. First, the xAI API and enterprise offerings are governed by their own terms, not the consumer privacy policy. Second, "the policy says X may train on your data" is a statement about what xAI permits itself to do, not proof of what happens to any given conversation. That distinction matters for every provider, not just xAI.

## What does xAI's privacy policy actually say?

The policy spans three distinct surfaces, and conflating them is where most bad takes come from.

### Consumer Grok: grok.com, the apps, and X

As of 2026-08-20, xAI's [privacy policy](https://x.ai/legal/privacy-policy) has allowed the company to use interactions with Grok (the text you send, the responses you receive, and explicit feedback like thumbs-up/down) to train, fine-tune, and improve its models. This is the same broad pattern most consumer chatbots started with: free and paid consumer tiers double as data-collection surfaces, and the burden is on you to opt out.

Retention is the second half of the question. Consumer policies typically retain conversation data for some period for safety, abuse prevention, and service operation even when you opt out of training. If you need a specific retention window, read the current policy text rather than relying on secondary coverage. xAI has revised these terms multiple times since Grok's launch, and the details move.

### Your X posts and Grok

xAI's structural advantage is X itself. X's privacy policy and related settings (see xAI's [privacy policy](https://x.ai/legal/privacy-policy) for the API side) have permitted public posts and interactions on the platform to be used for training Grok. X has stated that protected accounts and direct messages are excluded from this. Treat that as a policy commitment, not an audited fact, because no outside party has verified the pipeline.

European users have seen different treatment at times. Ireland's [Data Protection Commission](https://www.dpc.ie), X's lead EU regulator, previously intervened over the use of EU user data for Grok training, and X agreed to suspend that processing while the matter was addressed. The practical lesson: what the policy permits can vary by jurisdiction, and it can change after regulatory action, not just after a terms-of-service edit.

### The xAI API and enterprise tiers

If you call Grok through the xAI API, the consumer privacy policy is not the document that governs you. API usage falls under xAI's separate API terms, and enterprise deals are negotiated contracts. Across the industry, paid API tiers commonly carry stronger data-use terms than consumer apps, but "commonly" is doing a lot of work in that sentence. Read the current text on xAI's [legal page](https://x.ai/legal) and [API docs](https://docs.x.ai) before routing any sensitive workload. If you're an enterprise buyer, an explicit no-training clause is a standard ask; get it in writing rather than inferring it from a marketing page.

## How do you opt out of Grok training?

Menu labels shift, but as of this writing the opt-out lives in two places:

- **On X:** go to Settings → Privacy and safety → Data sharing and personalization, and look for the Grok data-sharing toggle. Turning it off tells X not to use your platform data and Grok interactions for training.
- **On grok.com and the standalone apps:** open Settings and look for data controls or a training opt-out. xAI has moved these controls between releases, so if the path above doesn't match what you see, check the current help documentation.

Three caveats practitioners should internalize. First, opt-outs are prospective: they stop future use and don't reach back into checkpoints already trained. Second, deleting a conversation from your history is a UI action, not necessarily a revocation of training rights already exercised. Third, opting out of training is not the same as opting out of retention: the provider may still hold your logs for safety and abuse-prevention purposes under the same policy.

## How does Grok's data policy compare to other providers?

On default posture, xAI sits in the industry mainstream rather than at either extreme. The table below summarizes each provider's *published* position as of 2026-08-20. All four revise these documents regularly, and "published policy" is not the same as "verified practice."

| Provider | Consumer chat default (per policy) | API default (per policy) | Opt-out mechanism |
|---|---|---|---|
| xAI (Grok) | Training allowed unless you opt out | Separate API terms; read before sending sensitive data | Settings toggle |
| OpenAI (ChatGPT) | May use content unless training is disabled | Says API inputs aren't used for training by default | Settings / per-chat controls |
| Anthropic (Claude) | Says it doesn't train on inputs without permission | Same stated posture on API | Feedback opt-in |
| Google (Gemini) | App activity may be used, depending on settings | Paid API tiers carry no-training commitments | Activity controls |

The honest takeaway: no major consumer chatbot offers strong privacy by default, and the differences are in opt-out friction and API terms, not in kind. If your threat model is "the provider must not see this at all," none of these rows is your answer.

## Why does Grok's data policy keep changing?

Three forces drive the churn. Regulation is the biggest: EU data-protection authorities have already forced xAI to alter its training practices once, and further rulings will likely do so again. Product tiering is the second: every new subscription tier, API plan, or enterprise offering gets its own terms, and the boundaries between them shift. The third is industry norm pressure: when a competitor tightens its defaults, others tend to follow within a release cycle or two.

For anyone building on Grok, the operational answer is to track the policy like a dependency. Pin the version of the terms you accepted, re-read on each revision notice, and keep a dated record of the posture you relied on. DeAI's [provider trust hub](/trust) tracks policy changes like these across inference providers so you don't have to diff legal PDFs yourself.

## What are your options if you need stronger privacy?

Match the guarantee to the sensitivity of the data:

1. **Contractual no-training terms.** If you're a business, this is the baseline: an enterprise agreement that explicitly prohibits training on your inputs and specifies retention. A sales page is not a contract.
2. **Self-host open weights.** Running a model on your own hardware removes the provider from the data path entirely. It's the only architecture with zero third-party exposure by construction. Notably, xAI released the original Grok-1 weights under Apache-2.0 (see the [model card](https://huggingface.co/xai-org/grok-1)), though it's a very large mixture-of-experts model that's impractical for most individuals to serve, and newer Grok versions have not been open-weight. The broader open-weight ecosystem offers more tractable options.
3. **Zero-retention providers.** Several inference providers advertise zero data retention. Treat "zero retention" as a policy claim unless it's backed by an attestation or audit you can read. Most aren't.
4. **Decentralized inference marketplaces.** Morpheus is one example: prompts are routed to independent node operators rather than a single provider's data center. Morpheus states that operators can't see prompt contents in identifying form; as with any privacy absence-claim, treat that as an architectural and policy assertion, not a verified fact.

The rule of thumb: if leaking the prompt would cost you money, a customer, or a legal privilege, don't send it to any consumer chatbot, Grok included, regardless of what the toggle says.

## FAQ

### Does xAI's Grok train on your data?

Yes, by default on consumer products: xAI's privacy policy has allowed it to use Grok conversations and X platform data to train models unless you opt out. API and enterprise tiers are governed by separate terms. Verify against the current policy, which changes often.

### How do I stop Grok from training on my data?

On X, use the Grok data-sharing toggle in privacy settings; grok.com and the apps have an equivalent opt-out in settings. Opt-outs apply going forward and don't erase data already used. Menus change often, so check xAI's current help pages.

### Does the xAI API train on my prompts?

API usage is covered by xAI's separate API and enterprise terms, which differ from the consumer privacy policy. Read the current API terms before sending sensitive data; enterprise customers can typically negotiate explicit no-training terms.

### Is Grok private enough for sensitive work?

Treat consumer Grok like any cloud chatbot that may train on inputs: don't paste confidential data unless your tier has contractual no-training terms. For stronger guarantees, use local open-weight models or providers with zero-retention policies.

## FAQ

**Does xAI's Grok train on your data?**

Yes, by default on consumer products: xAI's privacy policy has allowed it to use Grok conversations and X platform data to train models unless you opt out. API and enterprise tiers are governed by separate terms. Verify against the current policy, which changes often.

**How do I stop Grok from training on my data?**

On X, use the Grok data-sharing toggle in privacy settings; grok.com and the apps have an equivalent opt-out in settings. Opt-outs apply going forward and don't erase data already used. Menus change often — check xAI's current help pages.

**Does the xAI API train on my prompts?**

API usage is covered by xAI's separate API and enterprise terms, which differ from the consumer privacy policy. Read the current API terms before sending sensitive data; enterprise customers can typically negotiate explicit no-training terms.

**Is Grok private enough for sensitive work?**

Treat consumer Grok like any cloud chatbot that may train on inputs: don't paste confidential data unless your tier has contractual no-training terms. For stronger guarantees, use local open-weight models or providers with zero-retention policies.

## Sources

- [xAI Privacy Policy](https://x.ai/legal/privacy-policy) — xAI
- [xAI Legal & Policies](https://x.ai/legal) — xAI
- [xAI Privacy Policy](https://x.ai/legal/privacy-policy) — X Corp
- [xAI API Documentation](https://docs.x.ai) — xAI
- [Data Protection Commission (Ireland)](https://www.dpc.ie) — DPC
- [Grok-1 Model Card](https://huggingface.co/xai-org/grok-1) — Hugging Face


---

# GPT-5.5 vs Open Models in 2026: Can DeepSeek V4, Kimi K3 Replace It?

*By DeAI Newsroom · 2026-08-24 · 8 min read*

Canonical: https://www.deai.org/news/gpt-5-5-vs-open-models

> **Summary:** As of August 2026, open-weight models like DeepSeek V4 and Kimi K3 can replace GPT-5.5 for most production workloads, typically at a fraction of frontier-API pricing. GPT-5.5's remaining moat is the hardest reasoning, long-horizon agentic, and multimodal work.

Yes, for most workloads. As of August 2026, open-weight models like DeepSeek V4 and Kimi K3 handle the bulk of chat, coding, and RAG traffic that teams used to send to GPT-5.5, typically at a fraction of frontier-API pricing. GPT-5.5 keeps its edge on a shrinking slice of tasks. Four questions tell you which slice you're in.

## Key takeaways

- Open-weight models now cover the majority of standard production traffic (chat, summarization, RAG, routine coding), with GPT-5.5's clear edge concentrated in 2 areas: hardest-tier reasoning and long-horizon agentic or multimodal work.
- The frontier-vs-open lag has compressed from over a year to a matter of months on many public benchmarks, but leaderboard averages hide wide per-task variance.
- Open-weight inference typically costs a fraction of frontier-API pricing; prices move monthly, so check each provider's pricing page before modeling savings.
- Migration is usually a 1-line base-URL swap against an OpenAI-compatible endpoint, plus an eval pass, not a rewrite.
- The durable 2026 pattern is routing: 1 frontier model for the hard tail, open weights for everything else.

## Is any open model as good as GPT-5.5?

It's the question behind every "GPT-5.5 vs open source models" search, and it has a cleaner answer than most comparisons admit: as good at what?

Split your traffic into two zones. The **parity zone** covers customer chat, summarization, extraction, RAG question-answering, translation, and routine code completion and review. On these tasks, leading open-weight models are effectively interchangeable with GPT-5.5 as of August 2026; differences show up at the margins of style and refusal behavior, not in task success. The **frontier zone** covers competition-grade math and reasoning, long-horizon agentic tasks with dozens of dependent tool calls, ambiguous instruction-following, and multimodal inputs. Here GPT-5.5 still earns its premium.

There are also axes where "as good" has nothing to do with quality. Open weights win outright on deployment control: you can run them in your own VPC or on-prem, keep prompts inside your perimeter, fine-tune without a vendor's pipeline, and escape per-call rate limits. Closed wins on the managed wrapper: compliance paperwork, SLAs, and a mature tooling ecosystem. If your traffic is 80% parity-zone tasks, the honest answer to "is it as good" is yes, for you.

## How big is the frontier-vs-open gap in 2026?

Narrow, and narrower than it was. In 2023, open-weight models trailed the frontier by well over a year. Through 2025 and into 2026, that lag compressed to months on many public benchmarks, and some older benchmarks have effectively saturated: everyone scores near the ceiling, so the benchmark stops discriminating.

Three caveats before you read any leaderboard. First, vendor-published numbers are claims: DeepSeek's and Moonshot's published evals position their models as frontier-adjacent, and OpenAI's published evals position GPT-5.5 as clearly ahead. Both are self-reported; treat them as marketing until triangulated. Second, contamination is real: test sets leak into training data, which inflates scores on exactly the benchmarks buyers cite. Third, averages hide variance: a model can match the frontier on a benchmark mean while failing far more often on adversarial or unusual inputs, and it's the tail that pages you at 3 a.m.

For independent reads, cross-reference human-preference leaderboards like LMArena with task-focused trackers like Artificial Analysis, and weight your own golden eval set above both. Axes most public leaderboards ignore entirely, like how often a model refuses benign requests, matter for production too; DeAI's refusal-index methodology scores that dimension precisely because standard benchmarks don't.

Where the gap genuinely persists: long-horizon agency (reliability compounds per step, so small per-step differences become large end-to-end ones), multimodal maturity, and consistency under distribution shift.

## What can DeepSeek V4 replace?

DeepSeek's V3 and R1 releases made the family the default open answer for reasoning-heavy work, and V4 continues that line: the company positions it as frontier-competitive on reasoning, math, and coding, with the mixture-of-experts cost profile that made its predecessors cheap to serve. Those are DeepSeek's claims; verify against your tasks.

In practice, DeepSeek's line is the natural first candidate for backend and batch coding jobs, math-heavy pipelines, agentic loops where outputs are machine-verifiable, and any high-volume traffic where frontier-API pricing is the line item killing your margins. Weights are published on Hugging Face, and the first-party API is documented at DeepSeek's developer site. Check the model card for license terms, context-window specifics, and tokenizer behavior before committing, and watch reasoning-token verbosity: long chains of thought inflate both latency and bill.

## What can Kimi K3 replace?

Moonshot AI's Kimi line built its reputation on long context and agentic tool use. K2 was the open-weight model that made "agentic" a credible open-model adjective, and K3 is positioned as its successor. Again, Moonshot's published numbers are claims.

The Kimi line fits workloads where context length is the binding constraint: long-document summarization and contract review, repo-scale code Q&A, and multi-step workflows that carry state across many tool calls. If your GPT-5.5 usage is dominated by stuffing large documents into the prompt, K3 belongs on your shortlist alongside DeepSeek V4, with the same due-diligence checklist: model card, license, and a golden-set eval before any traffic moves.

## Where does GPT-5.5 still win?

Give the closed model its due. GPT-5.5 remains the safer default for the hardest reasoning problems, for long-horizon agents where a 2% per-step reliability edge compounds into a large end-to-end gap, and for multimodal pipelines where open alternatives are less battle-tested. The enterprise wrapper matters too: SLAs, compliance attestations, batch and fine-tuning tooling, and a deep integrations ecosystem are real value that has nothing to do with benchmark scores.

OpenAI's own published evals are self-reported like everyone else's, but the broad picture, a real but narrowing frontier edge, is corroborated by independent leaderboards. The practical question isn't whether GPT-5.5 is better; it's whether your traffic actually touches the tasks where it's better.

## What are the best GPT-5.5 alternatives?

The shortlist starts with the two headliners (DeepSeek V4 for reasoning and code, Kimi K3 for long context and agents) and rounds out with Alibaba's Qwen family, Meta's Llama line, and Mistral's releases, all of which ship competitive open-weight generations worth benchmarking on your tasks. DeAI's [model directory](/models) tracks current open-weight releases and where they run.

On where to run them, evaluate every option on identical criteria: price per token, latency and region, data-retention policy, and model availability. Your choices are the first-party APIs (DeepSeek, Moonshot), hosted inference providers (Together, Fireworks, OpenRouter), Morpheus, a decentralized inference marketplace, or self-hosting on your own GPUs. One honesty note on privacy: zero-retention and "operators can't see prompts" statements from any provider are policy claims, not independently verified facts. If data control is a hard requirement rather than a preference, self-hosting is the only option that doesn't require trusting someone's policy page.

## How do you migrate from GPT-5.5 to an open model?

The mechanics are trivial; the evals are the work. Inventory your prompts by task, build a golden set of a few hundred representative inputs with known-good outputs, then run the swap in shadow mode before routing live traffic. Most open-model providers expose OpenAI-compatible endpoints, so the client change is a base URL and key:

```python
from openai import OpenAI
import os

client = OpenAI(
    api_key=os.environ["PROVIDER_API_KEY"],
    base_url=os.environ["PROVIDER_BASE_URL"],  # e.g. your provider's /v1 endpoint
)

resp = client.chat.completions.create(
    model="your-open-model-name",  # the provider's DeepSeek V4 or Kimi K3 identifier
    messages=[{"role": "user", "content": "Summarize this ticket in one line."}],
)
print(resp.choices[0].message.content)
```

```bash
curl "$PROVIDER_BASE_URL/chat/completions" \
  -H "Authorization: Bearer $PROVIDER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "your-open-model-name",
    "messages": [{"role": "user", "content": "Summarize this ticket in one line."}]
  }'
```

Budget your time for the differences the code sample hides: tool-calling schemas vary between families, system-prompt handling and refusal patterns differ, `max_tokens` semantics aren't universal, and prompts tuned for GPT-5.5's instruction-following habits often need re-tuning. DeAI's full [migration guide](/migrate-from-openai) walks through the eval harness and routing setup. For most teams the end state is a router rather than a single replacement: open weights for the parity zone, GPT-5.5 for the frontier tail.

## FAQ

### What are the best GPT-5.5 alternatives in 2026?

DeepSeek V4 and Kimi K3 are the most credible open-weight GPT-5.5 alternatives for reasoning, coding, and long-context work, with Alibaba's Qwen and Meta's Llama families rounding out the shortlist. Most expose OpenAI-compatible endpoints, so switching is a config change, not a rewrite.

### Is any open model as good as GPT-5.5?

On most everyday production tasks (chat, summarization, RAG, routine coding), leading open-weight models are effectively at parity with GPT-5.5 as of August 2026. GPT-5.5 still leads on the hardest reasoning, long-horizon agentic tasks, and multimodal work. As good depends on your task mix.

### How big is the frontier vs open gap in 2026?

Narrow and task-dependent. On many public benchmarks the open-weight lag has compressed to months, but averages hide variance: the gap widens on long-horizon agentic work, multimodal inputs, and adversarial edge cases. Evaluate on your own data, not leaderboard averages.

### Can I switch from GPT-5.5 to an open model without rewriting my app?

Usually yes. Major open-model providers expose OpenAI-compatible chat endpoints, so migration is typically a base-URL and API-key swap plus a model-name change. Budget time for evals: prompt behavior, refusal patterns, and tool-calling formats differ across model families.

## FAQ

**What are the best GPT-5.5 alternatives in 2026?**

DeepSeek V4 and Kimi K3 are the most credible open-weight GPT-5.5 alternatives for reasoning, coding, and long-context work, with Alibaba's Qwen and Meta's Llama families rounding out the shortlist. Most expose OpenAI-compatible endpoints, so switching is a config change, not a rewrite.

**Is any open model as good as GPT-5.5?**

On most everyday production tasks — chat, summarization, RAG, routine coding — leading open-weight models are effectively at parity with GPT-5.5 as of August 2026. GPT-5.5 still leads on the hardest reasoning, long-horizon agentic tasks, and multimodal work. As good depends on your task mix.

**How big is the frontier vs open gap in 2026?**

Narrow and task-dependent. On many public benchmarks the open-weight lag has compressed to months, but averages hide variance: the gap widens on long-horizon agentic work, multimodal inputs, and adversarial edge cases. Evaluate on your own data, not leaderboard averages.

**Can I switch from GPT-5.5 to an open model without rewriting my app?**

Usually yes. Major open-model providers expose OpenAI-compatible chat endpoints, so migration is typically a base-URL and API-key swap plus a model-name change. Budget time for evals: prompt behavior, refusal patterns, and tool-calling formats differ across model families.

## Sources

- [OpenAI API documentation — Models](https://platform.openai.com/docs/models) — OpenAI
- [OpenAI API pricing](https://openai.com/api/pricing/) — OpenAI
- [DeepSeek API documentation](https://api-docs.deepseek.com) — DeepSeek
- [DeepSeek model cards on Hugging Face](https://huggingface.co/deepseek-ai) — DeepSeek
- [Moonshot AI model cards on Hugging Face](https://huggingface.co/moonshotai) — Moonshot AI
- [Moonshot AI platform](https://platform.moonshot.ai) — Moonshot AI
- [LMArena leaderboard](https://lmarena.ai) — LMArena
- [Artificial Analysis model benchmarks](https://artificialanalysis.ai) — Artificial Analysis


---

# Leaving the Anthropic API: Open-Model Equivalents & the Switch (2026)

*By DeAI Newsroom · 2026-08-24 · 8 min read*

Canonical: https://www.deai.org/news/migrate-from-anthropic

> **Summary:** Migrating off the Anthropic API is mostly a base-URL swap plus a model shortlist: Qwen3, DeepSeek, and Llama cover most Claude workloads. Hosted zero-retention promises are policy statements; self-hosting is the only verifiable path.

Leaving the Anthropic API rarely requires a rewrite: for most teams, the switch comes down to one line of configuration (the base URL) plus picking an open-weight model from a shortlist of three families. This guide maps Claude workloads to open equivalents and walks the migration.

## Key takeaways

- The switch is usually one line of configuration (a new base URL) if your code already speaks the OpenAI schema; Anthropic-only features (prompt caching, extended thinking) are where the real work sits.
- Three open-model families cover most Claude workloads: Qwen3, DeepSeek, and Llama. Kimi K2 and OpenAI's gpt-oss are credible fourth options.
- Exactly one deployment model gives you verifiable zero retention: self-hosting on hardware you control. Hosted "zero-retention" is a policy statement, not an auditable fact.
- Plan two rollout phases: shadow mode against live traffic, then endpoint-by-endpoint cutover, with a rollback flag throughout.
- Budget four checkpoints: prompt inventory, model shortlist, a small eval on your own data, staged rollout.

## Why are teams re-evaluating the Anthropic API in 2026?

In 2025 Anthropic updated its consumer privacy policy so that claude.ai chats could be used for training unless users opted out, with longer retention for those who allow it. The change triggered days of retention-policy anxiety on X, and a wave of conflation between Anthropic's consumer terms and its API terms, which are separate documents with different commitments.

Worth stating plainly: Anthropic's commercial terms have historically been among the clearer ones in the industry, stating that API inputs are not used to train models by default. This is not a dunk piece. The structural lesson is older than any one vendor: any hosted provider's terms are unilateral and changeable. If your prompts, your users' data, or your compliance posture can't absorb a policy change you don't control, the answer is optionality, not outrage. Cost at scale and rate-limit ceilings push teams in the same direction.

## What does Anthropic's retention policy actually say?

Two documents matter, and they say different things:

- **Commercial / API terms.** Anthropic's published commercial terms state that customer API content is not used to train models by default, and its trust documentation defines retention windows for safety and abuse monitoring. Enterprise customers can negotiate modified terms. Read the current versions; they have changed before.
- **Consumer privacy policy (claude.ai).** The 2025 update moved consumer chats to an opt-out model for training use, with extended retention for users who allow it. This is the document that drove the X discourse. It does not govern API traffic.

The deeper point: for any hosted provider, retention is a policy statement. You cannot audit a datacenter from the outside. DeAI's [trust framework](/trust) separates provider claims into "policy" (what they promise) and "verifiable" (what you can check). It's a useful lens whether you stay or go.

## Which open models are the closest Claude alternatives?

No single open model is "the open Claude." Match by workload, and verify against current model cards and public leaderboards. The frontier moves monthly.

### Coding and agentic work

Claude's reputation was built here. The open shortlist most teams start with: the Qwen3-Coder family, DeepSeek-V3 and its successors, and Zhipu's GLM coding variants. All three sit near the frontier for code generation and tool calling on current public leaderboards, and all are served behind OpenAI-compatible endpoints at multiple hosts.

### General chat and reasoning

Qwen3-235B (Apache-2.0) and DeepSeek-R1 (MIT) are the default picks for reasoning-heavy workloads; Llama 3.3 70B covers lighter general-assistant duty on much cheaper hardware; Kimi K2 has a strong following for agentic, tool-heavy flows; OpenAI's gpt-oss-120b is Apache-2.0 and sized to fit on a single high-end GPU. Licenses differ (Llama ships under its own community license), so check terms before commercial deployment.

### Long context

Several open models now advertise context windows in the hundreds of thousands of tokens. Advertised length and usable recall are different things; test with your actual documents before committing a RAG pipeline to any window claim.

## Where should you run open models?

### Hosted OpenAI-compatible providers

Together, Fireworks, DeepInfra, Groq, Cerebras, and aggregators like OpenRouter all serve the major open weights behind OpenAI-compatible endpoints, typically at a fraction of frontier-API pricing. Check their published pricing pages for current rates. Some advertise zero retention; treat that as a policy statement, per the lens above.

### Decentralized marketplaces

Morpheus, a decentralized inference marketplace, routes requests to independent operators rather than one company's datacenters. That distributes trust instead of concentrating it, but the same rule holds: unless you control the hardware, retention assurances are policy, not proof.

### Self-hosting

vLLM or SGLang on rented or owned GPUs for production; Ollama or llama.cpp for development and small deployments. This is the only setup where "nobody sees your prompts" is a fact rather than a claim. The trade: you own uptime, scaling, and ops. It pencils out at sustained volume or under hard compliance requirements.

## How do you actually switch from the Claude API?

### Step 1: Inventory your usage

Export or log a week of Anthropic traffic: which models, token volumes, and which features you actually exercise (tool use, prompt caching, extended thinking, vision). Most teams discover they use a fraction of the surface area.

### Step 2: Shortlist and eval

Pick two or three candidate models and run 50–200 representative prompts through both Claude and the candidates. Grade with a rubric or an LLM judge. Your own prompts beat any public benchmark for this decision.

### Step 3: Swap the base URL

Most open-model hosts expose the OpenAI chat-completions schema, so the client change is small:

```python
import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.your-provider.example/v1",  # your host's endpoint
    api_key=os.environ["PROVIDER_API_KEY"],
)

response = client.chat.completions.create(
    model="your-chosen-model",  # exact name varies by provider
    messages=[
        {"role": "system", "content": "You are a careful assistant."},
        {"role": "user", "content": "Summarize this diff."},
    ],
)
print(response.choices[0].message.content)
```

```bash
curl https://api.your-provider.example/v1/chat/completions \
  -H "Authorization: Bearer $PROVIDER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "your-chosen-model",
    "messages": [
      {"role": "system", "content": "You are a careful assistant."},
      {"role": "user", "content": "Summarize this diff."}
    ]
  }'
```

If you're coming from the Anthropic SDK rather than raw HTTP, the mapping is well-trodden: Anthropic itself publishes an OpenAI-SDK compatibility shim, which is a decent reference for how the schemas line up. The same swap, from the other direction, is covered in our [guide to leaving the OpenAI API](/migrate-from-openai).

### Step 4: Port the Anthropic-specific features

- **System prompts**: Anthropic takes `system` as a top-level parameter; the OpenAI schema uses a system message. Trivial.
- **`max_tokens`**: required on Anthropic, optional in the OpenAI schema. Set sane defaults anyway.
- **Tool use**: Anthropic's `input_schema` maps to `parameters` under OpenAI `tools`. Both are JSON Schema; most definitions port mechanically, but test argument-validation edge cases.
- **Extended thinking**: reasoning-style open models (DeepSeek-R1, Qwen3 thinking variants) expose chain-of-thought differently: some providers return a separate `reasoning_content` field, some stream it inline. Normalize it in one wrapper function.
- **Prompt caching**: Anthropic uses explicit `cache_control` breakpoints. Many OpenAI-compatible hosts do automatic prefix caching, or none at all. Cost and latency behavior will differ; measure, don't assume.

### Step 5: Shadow, then cut over

Run the new model in shadow mode against live traffic for a few days, diff the outputs, then cut over endpoint by endpoint behind a feature flag. Keep Claude wired as a fallback until the new path has survived a full traffic cycle.

## What breaks when you leave Claude?

Honestly: some things. Claude's long-context recall and tool-use reliability are genuine strengths, and per-task gaps remain even as open models close the broad ones, which is why the Step 2 eval matters more than any article, including this one. Refusal behavior also differs across models and providers; DeAI's refusal-index methodology scores how models handle benign-but-edgy requests, and it is worth consulting as results publish.

Your MCP servers carry over. MCP is an open protocol, not an Anthropic product feature. What you gain in exchange for the gaps: portability across hosts, price competition at the model layer, freedom to fine-tune, and, if you self-host, an end to retention-policy risk as a category.

## FAQ

### Claude API data retention: what does Anthropic actually keep?

Anthropic's commercial terms state that API inputs are not used for training by default, with retention windows defined in its trust documentation. Consumer (claude.ai) terms differ and changed in 2025. Policies evolve, so read the current pages before deciding.

### What is the best open-source Claude alternative?

There is no single best. For coding and agentic work, Qwen3-Coder and DeepSeek-V3 are common starting points; for general chat, Qwen3-235B, Llama 3.3 70B, and Kimi K2. Run a small eval on your own prompts before committing.

### Do I have to rewrite my app to leave the Claude API?

Usually not. Most open-model hosts expose OpenAI-compatible endpoints, so the client change is a base URL, an API key, and a model name. Anthropic-specific features (tool schemas, prompt caching, extended thinking) need targeted porting.

### Is self-hosting the only way to get verifiable zero retention?

Yes, if "verifiable" is the requirement. A hosted provider's zero-retention promise is a policy statement you cannot audit from outside. Self-hosting on hardware you control is the only setup where "nobody sees your prompts" is a fact, not a claim.

## FAQ

**Claude API data retention: what does Anthropic actually keep?**

Anthropic's commercial terms state that API inputs are not used for training by default, with retention windows defined in its trust documentation. Consumer (claude.ai) terms differ and changed in 2025. Policies evolve — read the current pages before deciding.

**What is the best open-source Claude alternative?**

There is no single best. For coding and agentic work, Qwen3-Coder and DeepSeek-V3 are common starting points; for general chat, Qwen3-235B, Llama 3.3 70B, and Kimi K2. Run a small eval on your own prompts before committing.

**Do I have to rewrite my app to leave the Claude API?**

Usually not. Most open-model hosts expose OpenAI-compatible endpoints, so the client change is a base URL, an API key, and a model name. Anthropic-specific features — tool schemas, prompt caching, extended thinking — need targeted porting.

**Is self-hosting the only way to get verifiable zero retention?**

Yes, if 'verifiable' is the requirement. A hosted provider's zero-retention promise is a policy statement you cannot audit from outside. Self-hosting on hardware you control is the only setup where 'nobody sees your prompts' is a fact, not a claim.

## Sources

- [Anthropic Privacy Policy](https://www.anthropic.com/legal/privacy) — Anthropic
- [Anthropic Commercial Terms of Service](https://www.anthropic.com/legal/commercial-terms) — Anthropic
- [Anthropic API Documentation](https://docs.anthropic.com) — Anthropic
- [Qwen3-235B-A22B model card](https://huggingface.co/Qwen/Qwen3-235B-A22B) — Hugging Face
- [DeepSeek-V3 model card](https://huggingface.co/deepseek-ai/DeepSeek-V3) — Hugging Face
- [Llama 3.3 70B Instruct model card](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct) — Hugging Face
- [Kimi K2 Instruct model card](https://huggingface.co/moonshotai/Kimi-K2-Instruct) — Hugging Face
- [vLLM Documentation](https://docs.vllm.ai) — vLLM Project


---

# Migrate Off the OpenAI API in an Afternoon (2026 — Code Included)

*By DeAI Newsroom · 2026-08-24 · 9 min read*

Canonical: https://www.deai.org/news/migrate-from-openai

> **Summary:** Migrating off the OpenAI API is a three-line code change: new base URL, new key, new model name — your existing SDK keeps working. The safe rollout is a 5% traffic canary for two hours with one-flag rollback, so the commitment is zero until the data says otherwise.

You can migrate from the OpenAI API to an open-weight model provider in one afternoon, and the code change is three lines: a new base URL, a new API key, and a new model name. The safe rollout pattern is a canary deploy: 5% of traffic for two hours, with instant rollback and zero commitment.

## Key takeaways

- The migration is a **3-line change** (`base_url`, `api_key`, model name), and your existing `openai` SDK keeps working against any OpenAI-compatible endpoint.
- Canary **5% of traffic for 2 hours** before cutover; rollback is a single environment-variable flip, so the commitment is zero.
- A realistic plan fits in an afternoon: ~1 hour picking a provider and model, ~1 hour validating a golden prompt set, 2 hours of canary.
- Chat completions port cleanly; **embeddings do not**. Re-embedding your corpus is the one task that can blow the afternoon budget.
- Open-weight endpoints are typically a fraction of frontier-API pricing, but verify on each provider's published pricing page rather than trusting any summary, including this one.

## Why migrate from the OpenAI API at all?

Teams that migrate rarely do it because of a single dramatic incident. The common drivers are structural. Portability: when your workload runs against a standardized interface, no single vendor's pricing or deprecation schedule is an emergency anymore. Model choice: open-weight families (DeepSeek, Meta's Llama, Alibaba's Qwen, Mistral, Zhipu's GLM) now cover most production workloads, and you can run the same weights with multiple providers or on your own hardware. Cost structure: open-weight serving is typically a fraction of frontier-API pricing, though you should confirm current numbers on each provider's pricing page. And data handling: some teams need weights they can run inside their own perimeter for contractual or regulatory reasons.

None of this means frontier APIs stop being the right tool for some tasks. For a capability framing of GPT-5.5-class frontier models against open weights, see [GPT-5.5 vs. open models](/gpt-5-5-vs-open-models). The point of migrating is optionality, not declaring a winner.

## What does "OpenAI-compatible" actually mean?

The `chat/completions` schema (a `messages` array, a `model` string, and `choices[0].message.content` in the response, with server-sent events for streaming) became the de facto industry interface. An "OpenAI-compatible" provider implements that same contract, which means the official `openai` Python and Node libraries work against it unchanged; the libraries simply accept a `base_url` parameter. Our explainer on the [OpenAI-compatible API](/openai-compatible-api) covers the contract in detail.

Providers fall into a few categories, all worth evaluating on identical criteria: model catalog, uptime track record, price per token, and stated retention policy.

| Provider | `base_url` | Category |
|---|---|---|
| DeepSeek | `https://api.deepseek.com` | First-party model API |
| OpenRouter | `https://openrouter.ai/api/v1` | Aggregator across many models |
| Together AI | `https://api.together.xyz/v1` | Hosted open-weight inference |
| Fireworks AI | `https://api.fireworks.ai/inference/v1` | Hosted open-weight inference |
| Groq | `https://api.groq.com/openai/v1` | Hosted inference, custom silicon |
| Self-hosted vLLM | `http://localhost:8000/v1` | Your own GPUs, your weights |

Decentralized inference marketplaces are a fourth category: Morpheus is one example, routing the same OpenAI-compatible calls to independent operators. Evaluate them on the same four criteria as any hosted provider. Where a provider states a zero-retention policy, treat that as a policy statement to verify contractually, not as an independently verified fact.

## What is the three-line base-URL swap?

This is the whole code change. Keep the `openai` package installed: you are not replacing your SDK, just pointing it somewhere else.

```python
import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["PROVIDER_API_KEY"],              # line 1: new key
    base_url=os.environ["PROVIDER_BASE_URL"],            # line 2: new endpoint
)

resp = client.chat.completions.create(
    model=os.environ.get("MODEL_NAME", "deepseek-chat"), # line 3: new model
    messages=[{"role": "user", "content": "Summarize this ticket."}],
)
print(resp.choices[0].message.content)
```

The same call as raw HTTP, useful for smoke-testing any endpoint before you touch application code:

```bash
curl "$PROVIDER_BASE_URL/chat/completions" \
  -H "Authorization: Bearer $PROVIDER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-chat",
    "messages": [{"role": "user", "content": "Hello"}]
  }'
```

Two details matter here. First, everything is driven by environment variables, which is what makes the canary deploy below a configuration change rather than a code change. Second, keep your old OpenAI credentials live during the migration. You are adding a backend, not deleting one. The [DeepSeek API docs](https://api-docs.deepseek.com/) and the [openai-python README](https://github.com/openai/openai-python) both document this pattern; aggregators like [OpenRouter](https://openrouter.ai/docs) and hosted providers like [Together AI](https://docs.together.ai/) publish the same `base_url` instructions.

## How do you replace GPT with an open model?

Model-name remapping is where migrations actually succeed or fail, because "GPT-class" is a workload description, not a model. Map by what the endpoint does:

| Your OpenAI workload | Open-model families to try first |
|---|---|
| General assistant / chat | DeepSeek V-line, Meta Llama, Alibaba Qwen, Mistral Large |
| Reasoning-heavy (math, planning, agents) | DeepSeek R-line, Qwen reasoning variants |
| Code generation and review | DeepSeek, Qwen Coder line |
| Multilingual workloads | Qwen, GLM |
| Tool-calling / function-calling loops | Verify per provider — support and strictness vary |

Pick one candidate per workload, then build a golden set: 30–50 real prompts from your production logs with the outputs you currently consider acceptable. Run the set against both backends and diff. This is the hour of work that separates a migration from a hope. Model cards on [Hugging Face](https://huggingface.co/deepseek-ai) document context windows, licenses, and intended use for each family. For the broader frontier-vs-open trade-off, see our [GPT-5.5 vs. open models](/gpt-5-5-vs-open-models) piece.

## How does the 5% canary deploy work?

The canary pattern is: route a small, stable slice of production traffic to the new backend, watch it for two hours, and keep the old path one config flip away. Zero commitment means exactly that: you have changed nothing irreversible until you decide to.

A sticky router, so the same users consistently hit the same backend:

```python
import hashlib

CANARY_PERCENT = 5  # two-hour canary window

def pick_backend(user_id: str) -> str:
    bucket = int(hashlib.sha256(user_id.encode()).hexdigest(), 16) % 100
    return "open-model" if bucket < CANARY_PERCENT else "openai"
```

During the two-hour window, watch four signals:

1. **Error rate**: HTTP failures and malformed responses on the canary backend versus baseline.
2. **Latency**: p50 and p95; a model that is right but slow will surface here first.
3. **Output quality**: re-run your golden set against live canary traffic patterns and spot-check a sample of real responses.
4. **Behavioral diffs**: `finish_reason` distribution, refusal frequency, and tool-call success rate. Refusal behavior varies noticeably between model families; DeAI's refusal-index methodology scores exactly this dimension across a standardized prompt set, and your golden set should include your own borderline prompts.

If anything is off, set `CANARY_PERCENT=0` (or repoint the env vars) and you are back on OpenAI in seconds. If the two hours are clean, ramp on your own schedule: 5% to 25% to 50% to 100% over the following days, watching the same four signals at each step.

## What breaks when you migrate off the OpenAI API?

The chat-completion swap is boring. These are the edges that are not:

- **Embeddings.** Vector dimensions and embedding spaces differ across models. You cannot mix old and new vectors in one index; you must re-embed the corpus. This is the single task most likely to exceed the afternoon.
- **Tool calling.** Schema strictness and parallel-tool-call behavior vary by provider and model. Test every function in your registry against the new backend.
- **JSON mode / structured outputs.** Support differs per endpoint; verify constrained generation against your schemas before the canary, not after.
- **Streaming edge cases.** Chunk boundaries and usage-reporting fields differ slightly between providers. If you parse SSE streams manually, test with real long outputs.
- **Rate limits and quotas.** Defaults differ; a traffic pattern that was fine on one backend can 429 on another. Load-test before ramping past 5%.
- **Tokenizers.** Token counts for identical text differ across model families, so cost and context-window estimates shift. Recalibrate both.
- **Retention and privacy terms.** These are policy statements, not technical guarantees. If data handling drove the migration, get the terms in writing; self-hosting with something like [vLLM](https://docs.vllm.ai/) is the only option where the answer is fully under your control.

## What does the afternoon look like, hour by hour?

| Time | Task |
|---|---|
| 0:00–1:00 | Pick one provider and one model per workload; create API keys; smoke-test with the curl snippet |
| 1:00–2:00 | Wire the env-var config; run your 30–50 prompt golden set against both backends; fix tool-calling and JSON-mode diffs |
| 2:00–4:00 | Deploy the sticky router at 5%; watch error rate, p95 latency, output samples, refusal behavior |
| 4:00+ | Decide: roll back in seconds, hold at 5%, or start the 25→50→100 ramp |

The embeddings caveat from above applies: if your product depends on a vector index, schedule the re-embedding as its own workstream. Everything else fits between lunch and dinner.

## FAQ

### What is a good open-source OpenAI API alternative?

There is no single best one. Any provider serving open-weight models over an OpenAI-compatible endpoint works with a base-URL swap. Compare model catalog, uptime, price per token, and stated data-retention policy on each provider's docs before committing.

### How do I replace GPT with an open model?

Point your existing OpenAI SDK at the new provider's base URL, swap the model name (a DeepSeek, Llama, or Qwen model, for example), then canary 5% of traffic for two hours while comparing error rates, latency, and output quality before full cutover.

### How does an OpenAI to DeepSeek migration work?

DeepSeek's API is OpenAI-compatible: set `base_url` to `https://api.deepseek.com`, use a DeepSeek API key, and change the model to `deepseek-chat` or `deepseek-reasoner`. No SDK change is required; test tool calling and JSON mode against your own prompts.

### How long does it take to migrate off the OpenAI API?

The code change is three lines and takes minutes. A realistic afternoon plan: about one hour to pick a provider and model, one hour to validate against a golden prompt set, then a two-hour 5% canary before ramping toward 100%.

### Will my embeddings still work after migrating?

Not directly. Embedding models differ in dimensions and vector space, so you must re-embed your corpus with the new provider's embedding model. Budget time for that separately from the chat-completion swap, which is the fast part.

## FAQ

**What is a good open-source OpenAI API alternative?**

There is no single best one. Any provider serving open-weight models over an OpenAI-compatible endpoint works with a base-URL swap. Compare model catalog, uptime, price per token, and stated data-retention policy on each provider's docs before committing.

**How do I replace GPT with an open model?**

Point your existing OpenAI SDK at the new provider's base URL, swap the model name (a DeepSeek, Llama, or Qwen model, for example), then canary 5% of traffic for two hours while comparing error rates, latency, and output quality before full cutover.

**How does an OpenAI to DeepSeek migration work?**

DeepSeek's API is OpenAI-compatible: set base_url to https://api.deepseek.com, use a DeepSeek API key, and change the model to deepseek-chat or deepseek-reasoner. No SDK change is required; test tool calling and JSON mode against your own prompts.

**How long does it take to migrate off the OpenAI API?**

The code change is three lines and takes minutes. A realistic afternoon plan: about one hour to pick a provider and model, one hour to validate against a golden prompt set, then a two-hour 5% canary before ramping toward 100%.

**Will my embeddings still work after migrating?**

Not directly. Embedding models differ in dimensions and vector space, so you must re-embed your corpus with the new provider's embedding model. Budget time for that separately from the chat-completion swap, which is the fast part.

## Sources

- [DeepSeek API Docs](https://api-docs.deepseek.com/) — DeepSeek
- [openai-python (official OpenAI Python library)](https://github.com/openai/openai-python) — OpenAI
- [OpenRouter Documentation](https://openrouter.ai/docs) — OpenRouter
- [Together AI Documentation](https://docs.together.ai/) — Together AI
- [vLLM Documentation](https://docs.vllm.ai/) — vLLM Project
- [DeepSeek model cards on Hugging Face](https://huggingface.co/deepseek-ai) — Hugging Face


---

# From Ollama to a Private Endpoint: Keep Privacy, Drop Ops (2026)

*By DeAI Newsroom · 2026-08-24 · 9 min read*

Canonical: https://www.deai.org/news/ollama-to-private-endpoint

> **Summary:** Ollama is excellent for local development, but production self-hosting means owning drivers, uptime, and a port that should never face the public internet.

Moving from Ollama to a hosted private LLM endpoint takes one code change: swap the OpenAI-compatible base URL and add an API key. The real work is everything around that line: verifying a provider's retention policy, matching your models, and closing the security gaps self-hosting left behind, like an exposed port 11434.

## Key takeaways

- The migration is one config change: point your OpenAI-compatible client from `http://localhost:11434/v1` at your provider's endpoint and add an API key.
- Ollama binds to `127.0.0.1:11434` by default and ships with no built-in authentication, the two documented facts behind most exposed-instance incidents.
- Hosted privacy rests on two policy levers, data retention and training use, and both are provider claims to verify in writing, not guarantees.
- Budget the decision with three numbers: your GPU's fixed cost, your actual utilization, and the provider's published per-token price.
- Keep Ollama for the one job it still does best: offline development and prototyping on your own hardware.

## Why the Ollama ops burden bites in production

The pattern is a familiar one. You installed Ollama on a workstation or a spare GPU box, the demo worked, a teammate pointed their app at it, and, without anyone deciding it, that box became production infrastructure. Ollama is genuinely excellent at what it's designed for: running open-weight models locally with a single command. What it was never going to do is run itself.

Self-hosted LLM maintenance is a bursty, unglamorous workload. Quiet for weeks, then an NVIDIA driver or CUDA update breaks inference after a routine OS patch. A model pull fills the disk because old quantizations never got pruned. Someone raises the context length and the box starts OOM-killing under load. You want a second replica for availability, which means another GPU and a load balancer you now also own. Teams that outgrow Ollama's serving layer often graduate to vLLM for throughput and scheduling. That's a real capability jump, but it also increases the ops surface you're personally responsible for.

None of this is an argument that self-hosting is wrong. It's an argument that the maintenance bill arrives on its own schedule, and for a lot of small teams the bill eventually exceeds the value of keeping everything in-house.

## Is your Ollama instance exposed to the internet?

Before planning any move, check the security posture of what you're running today. Ollama's defaults are safe: it binds to `127.0.0.1`, so only local processes can reach it. The trouble starts when you want teammates or services on other machines to connect. Ollama's own FAQ documents how to expose it on your network (typically by setting `OLLAMA_HOST=0.0.0.0`), and Ollama ships without built-in authentication. Anyone who can reach port 11434 can run inference, list your models, and pull or delete them.

Security researchers scanning the internet have repeatedly flagged publicly reachable Ollama instances, and the cause is almost always the same: a convenience setting applied to a machine with a public IP, with no proxy in front of it. If you stay self-hosted, the hardening checklist is short but non-negotiable:

- Keep the localhost binding and reach the box through an SSH tunnel or a mesh network like Tailscale.
- If you must expose it, put a reverse proxy (nginx or Caddy) in front with TLS and authentication, plus firewall rules that allowlist known clients.
- Never expose port 11434 directly on a public interface.

Here's the connection that pushes people toward hosted endpoints: this hardening work *is* the ops burden. A hosted endpoint moves TLS, authentication, and patching to the provider. In exchange, you stop trusting your own network configuration and start trusting a provider's data policy. That trade is the subject of the next two sections.

## What is a hosted private LLM endpoint, exactly?

A hosted private LLM endpoint is a managed inference API (in this context, one serving open-weight models through an OpenAI-compatible interface) whose provider contractually commits to two things: not retaining your prompts and responses, and not training on your data. Those are the only two levers that define "private" at the API layer, and both deserve scrutiny.

Zero-retention and no-training commitments are **policy statements, not verified facts**. A provider saying it doesn't store your prompts is a claim about its own behavior; you can't audit its logs from the outside. Some providers publish detailed data-usage terms, differentiate retention by tier, or offer enterprise agreements with stronger language. Read those documents, not the marketing page. Our guide to [zero-retention AI APIs](/zero-retention-ai-apis) breaks down what the different policy formulations actually mean and which questions to ask.

Also distinguish "we don't store" from "we can't see." The first is a retention policy. The second is an architectural claim, and it's much rarer. For most hosted providers, prompts pass through infrastructure the provider controls in plaintext; the privacy you get is contractual. Whether that's sufficient depends on your threat model, which is a decision input, not a reason to dismiss the option.

## Should you stay self-hosted or move?

The economics hinge on utilization. A dedicated GPU costs the same whether it's serving requests or sitting idle at 3 a.m.; a per-token API costs nothing when you're not calling it. Self-hosting wins on cost when utilization is high and steady; hosted APIs win when demand is spiky or modest. Our [self-hosting vs. API cost framework](/self-hosting-vs-api-cost) walks through the breakeven math with the three numbers you need: your hardware's fixed cost, your measured utilization, and the provider's published per-token price.

Cost isn't the only axis:

- **Data residency and regulation.** Some workloads legally can't leave your infrastructure. That settles the question in favor of self-hosting, full stop.
- **Latency.** A model on your LAN responds differently than one across the internet. For interactive local tooling, local often feels better; for server-side workloads, the difference is usually smaller than expected.
- **Model control.** Your own fine-tune or an unusual quantization may not exist in any provider catalog.
- **Burst capacity.** Hosted endpoints absorb traffic spikes without you buying hardware for the peak.

## How do you pick an Ollama alternative API?

Evaluate every candidate on the same checklist, in this order:

1. **Retention and training policy.** Find the actual data-usage terms. Check whether commitments differ by tier or require an opt-in.
2. **OpenAI compatibility.** Chat completions, streaming, tool calling, and embeddings — verify the surface you use, not just the logo.
3. **Model catalog.** Does it serve the open-weight families you run today? Cross-check provider catalogs against model cards on Hugging Face.
4. **Throughput and context length.** Confirm the provider serves the context sizes your prompts need.
5. **Published pricing.** Per-token versus dedicated capacity; use each provider's pricing page, not third-party summaries.
6. **Jurisdiction.** Where inference happens matters for some compliance regimes.

The market sorts into a few categories, all worth evaluating against identical criteria:

- **General inference clouds** such as Together, Fireworks, and Groq serve large open-weight catalogs and publish data-usage policies.
- **Aggregators** like OpenRouter route requests to multiple backends, which means the applicable retention policy is the underlying provider's, so check per-route.
- **Privacy-positioned providers** such as Venice advertise minimal-storage policies; as with every provider, treat those as the company's own claims and read the terms.
- **Decentralized inference marketplaces** such as Morpheus route requests to independent compute providers; apply the same checklist, with extra attention to how policy commitments map onto a distributed operator set.

No category wins by default. The right answer is whichever provider's policy, catalog, and pricing survive your checklist.

## How do you migrate without breaking your app?

### Step 1: Inventory what you actually run

List every model, quantization, and context length in use; note whether you depend on embeddings, tool calling, or a front end like Open WebUI; and get a rough sense of daily token volume. Most teams discover they run fewer distinct models than they thought.

### Step 2: Map models to hosted equivalents

Match each local model to a hosted equivalent by family and size. Expect small output differences: providers serve their own precision and quantization choices, so a "same" model is rarely bit-identical to your local GGUF. If output stability matters, pin a small set of evaluation prompts and compare before cutting over.

### Step 3: Swap the client (the one-line change)

If you already use Ollama's OpenAI-compatible endpoint at `http://localhost:11434/v1`, your code is structurally ready. The migration is a base URL and an API key:

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://api.your-provider.com/v1",  # was http://localhost:11434/v1
    api_key="YOUR_API_KEY",
)

response = client.chat.completions.create(
    model="your-model-id",  # use the provider's exact model ID
    messages=[{"role": "user", "content": "Hello"}],
)
print(response.choices[0].message.content)
```

```bash
curl https://api.your-provider.com/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "your-model-id",
    "messages": [{"role": "user", "content": "Hello"}]
  }'
```

If you call Ollama's native API instead, the surface is similar but not identical. Embeddings in particular differ between Ollama's `/api/embeddings` and the OpenAI-style `/v1/embeddings`. Open WebUI users can add the hosted endpoint as an OpenAI-compatible connection in the admin settings and switch models per chat, which makes it a convenient shadow-run harness.

### Step 4: Lock down privacy settings before real traffic

Before sending production prompts, enable whatever zero-retention or no-training controls the provider exposes at the account or tier level, scope API keys per application, and confirm org-level defaults. Policy commitments only protect you if your account is actually configured under them.

### Step 5: Shadow-run, cut over, and close port 11434

Run the hosted endpoint in parallel with Ollama on a sample of real traffic. Compare outputs, latency, and cost. Then cut over, and finish the job: stop exposing Ollama, firewall the port, and keep your local install for what it's still best at.

## When should you keep Ollama?

Honestly: sometimes. Air-gapped or offline environments, strict data-control requirements, sunk hardware with high steady utilization, and local development are all strong reasons to stay. A hybrid pattern (Ollama for development and sensitive workloads, a hosted private endpoint for production and burst) is common and perfectly coherent. The goal was never to abandon self-hosting. It was to stop paying an ops tax you didn't choose.

## FAQ

### Can you use Ollama in production?

Yes, but you own uptime, GPU drivers, scaling, and hardening. Ollama ships without built-in authentication and binds to localhost by default, so production means adding a proxy, TLS, and monitoring yourself.

### What is a hosted private LLM endpoint?

A managed, usually OpenAI-compatible inference API whose provider commits, in its data-usage terms, not to retain your prompts or train on them. "Private" is a policy claim, so verify it in writing before migrating.

### How much maintenance does a self-hosted LLM need?

Expect driver and CUDA updates, model and quantization management, uptime monitoring, and network hardening. The load is sporadic but urgent. It clusters around incidents, which is what pushes many teams to hosted endpoints.

### Is a hosted endpoint as private as running Ollama locally?

Different, not equal. Local keeps data on hardware you physically control. Hosted privacy rests on the provider's zero-retention policy, a contractual claim, not a verified fact. Regulated workloads may still require self-hosting.

### What is the best Ollama alternative API?

There isn't one universal best. Compare OpenAI-compatible providers on identical criteria: retention policy, training use, model catalog, throughput, and published pricing, across inference clouds, aggregators, privacy-positioned providers, and decentralized marketplaces.

## FAQ

**Can you use Ollama in production?**

Yes — but you own uptime, GPU drivers, scaling, and hardening. Ollama ships without built-in authentication and binds to localhost by default, so production means adding a proxy, TLS, and monitoring yourself.

**What is a hosted private LLM endpoint?**

A managed, usually OpenAI-compatible inference API whose provider commits — in its data-usage terms — not to retain your prompts or train on them. 'Private' is a policy claim, so verify it in writing before migrating.

**How much maintenance does a self-hosted LLM need?**

Expect driver and CUDA updates, model and quantization management, uptime monitoring, and network hardening. The load is sporadic but urgent — it clusters around incidents, which is what pushes many teams to hosted endpoints.

**Is a hosted endpoint as private as running Ollama locally?**

Different, not equal. Local keeps data on hardware you physically control. Hosted privacy rests on the provider's zero-retention policy — a contractual claim, not a verified fact. Regulated workloads may still require self-hosting.

**What is the best Ollama alternative API?**

There isn't one universal best. Compare OpenAI-compatible providers on identical criteria: retention policy, training use, model catalog, throughput, and published pricing — across inference clouds, aggregators, privacy-positioned providers, and decentralized marketplaces.

## Sources

- [Ollama FAQ — exposing Ollama on your network](https://github.com/ollama/ollama/blob/main/docs/faq.md) — Ollama
- [Ollama OpenAI compatibility](https://ollama.com/blog/openai-compatibility) — Ollama
- [OpenAI Python API library](https://github.com/openai/openai-python) — OpenAI
- [Open WebUI documentation](https://docs.openwebui.com) — Open WebUI
- [Hugging Face model hub](https://huggingface.co/models) — Hugging Face
- [vLLM documentation](https://docs.vllm.ai) — vLLM
- [OpenRouter](https://openrouter.ai) — OpenRouter
- [Together AI](https://www.together.ai) — Together AI
- [Fireworks AI](https://fireworks.ai) — Fireworks AI
- [Groq](https://groq.com) — Groq
- [Venice](https://venice.ai) — Venice


---

# Open-Weight vs Open-Source AI Models: The Difference That Bites (2026)

*By DeAI Newsroom · 2026-08-24 · 9 min read*

Canonical: https://www.deai.org/news/open-weight-vs-open-source

> **Summary:** Open-weight models hand you the files; open-source models hand you the freedoms. Most of 2026's flagship releases are the former wearing the latter's clothes, and the conditions only trigger at scale, by geography, or on redistribution. Read the license file before it reads you.

Open-weight means you can download the model's weights. Open-source means you get the four freedoms to use, study, modify, and share it. In 2026, almost every frontier-adjacent release (DeepSeek, Kimi K3, Llama 4) is the former dressed as the latter, and the difference surfaces exactly once: when you scale.

## Key takeaways

- "Open weight" has no agreed legal definition; "open source" does: the OSI's Open Source AI Definition 1.0, built on 4 freedoms: use, study, modify, share.
- 3 of the biggest recent releases (Llama 4, Kimi K3, and pre-2026 Gemma generations) show how custom licenses attach conditions that trigger at scale, by geography, or on redistribution.
- Meta's Llama 4 license carves the EU out of multimodal use and keeps a 700 million monthly-active-user threshold above which you need a separate deal with Meta.
- Gemma 4's move to Apache 2.0 makes it the simplest commercial "yes" among 2026's major releases.
- The one number most custom licenses care about is your monthly active users — know yours before legal asks.

## What does "open weight" actually mean?

A model's weights are the learned parameter files — the thing you download from Hugging Face, load into vLLM or llama.cpp, and run on hardware you control. When a lab says a model is "open weight," it is describing a **distribution mechanism**: the files are public. It is not describing a set of rights.

That distinction matters because the rights you actually get live entirely in the license text attached to those files, and "open weight" has no standards body, no definition, and no floor. Two models can both be marketed as open-weight while giving you wildly different freedom: one under MIT, one under a multi-thousand-word custom agreement with user thresholds, naming duties, and regional carve-outs. The download button looks identical in both cases.

So treat "open weight" as a starting signal, not an answer. It tells you that you *can* self-host, quantize, and (usually) fine-tune. It tells you nothing about whether you may do those things commercially, in your jurisdiction, at your scale.

## What does "open source" actually mean?

"Open source" does have a definition. The Open Source Initiative, which has stewarded the Open Source Definition for software since 1998, released the [Open Source AI Definition (OSAID) 1.0](https://opensource.org/ai/open-source-ai-definition) in October 2024. It applies the classic four freedoms to AI systems: anyone may **use** the system for any purpose without permission, **study** how it works, **modify** it for any purpose, and **share** it with or without modifications.

The teeth are in the study freedom. The OSAID requires disclosure of sufficiently detailed information about the training data that a skilled person could build a substantially equivalent system. Almost no frontier-adjacent lab does this. That means that under the strict definition, even models with OSI-approved licenses on their weights (DeepSeek under MIT, Gemma 4 under Apache 2.0) fall short, because their training data remains undisclosed.

Practitioners mostly use a looser reading: weights under a recognized OSI-approved license (MIT, Apache 2.0) count as open source. Lawyers, standards folks, and the OSI itself use the strict one. Both positions are defensible. What is not defensible is using "open source" to describe a model under a bespoke license with commercial conditions, which is exactly what much of 2026's release marketing does.

## Why the difference bites later

The gap between the two terms is invisible on day one. You download the weights, run a demo, ship a feature. Nobody asks for the license file. The conditions in custom licenses are written to activate later, at four specific moments:

1. **Scale.** Threshold clauses key off your monthly active users or revenue. Cross the line and you owe the lab a conversation, and possibly a negotiated license.
2. **Acquisition or diligence.** An acquirer's counsel will read every model license in your stack. Ambiguous redistribution or derivative terms get priced into the deal, or flagged as a blocker.
3. **Geography.** Regional carve-outs sleep until you expand into the carved-out region, or hire a team there.
4. **Redistribution and derivatives.** Ship a fine-tune or distill a smaller model, and naming, attribution, and derivative-licensing clauses wake up.

None of these are hypothetical. They are the actual mechanics of the licenses behind the biggest open releases of the past two years.

## What are the Llama 4 license's EU restrictions?

Meta's [Llama 4 Community License](https://www.llama.com/llama4/license/) contains the most-discussed carve-out in recent releases: for the multimodal Llama 4 models, the license rights are not granted to you if you are an individual domiciled in, or a company with a principal place of business in, the European Union. The text-only models remain available to EU builders; the multimodal ones do not. The clause is widely read as Meta limiting its exposure to EU AI regulation. That is an interpretation, not a verified motive, but it is consistent with Meta's public complaints about regulatory unpredictability in the bloc.

Two more Llama conditions practitioners trip over:

- **The 700 million MAU clause.** Carried across Llama versions, it requires any entity whose products or services had more than 700 million monthly active users on the release date to request a separate license from Meta. Irrelevant for a startup; very relevant if you are building on top of a platform that already has scale.
- **Attribution and derivative naming.** Distribute Llama 4 or a derivative and you must display "Built with Llama," and derivative model names must begin with "Llama." Fine-tunes you ship to customers are distributions.

These terms attach to you, not to your infrastructure. Running Llama 4 through a US-hosted third-party API does not change where your company is domiciled.

## Kimi K3 license explained

Moonshot AI ships Kimi K3 under a **custom license**, not a standard OSI-approved text. That single fact should change how you evaluate it: there is no pre-existing legal playbook, and the launch blog post is not the binding document; the license file on the [model card](https://huggingface.co/moonshotai) is.

There is precedent for Moonshot adding conditions to otherwise permissive text: Kimi K2 used a modified MIT license that attached an attribution requirement for very large commercial deployments. Kimi K3's terms are their own document, so do not assume the K2 terms carry over in either direction. Before a commercial deployment, read the full license and check specifically for:

- The scope of permitted commercial use
- Any scale thresholds (monthly active users or revenue)
- Naming or attribution duties for products and derivatives
- Redistribution and fine-tune terms
- Termination and change-of-terms language

Once the license is cleared, the operational side is the easy part. Our [Kimi K3 API run guide](/run-kimi-k3-api) covers endpoints and setup.

## Gemma 4 and the Apache 2.0 reset

The sharpest contrast in 2026 comes from Google. Earlier Gemma generations shipped under a custom Gemma Terms of Use: workable, but another bespoke document for counsel to parse. Gemma 4 ships under [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0), the same license as Kubernetes and Kafka.

That matters more than it sounds. Apache 2.0 is OSI-approved, includes an express patent grant, and imposes no user thresholds, no geography clauses, and no naming duties. Every corporate legal department on earth already has a playbook for it. Under the strict OSAID, Gemma 4 still isn't fully "open source" (training-data disclosure remains the sticking point there), but as a pure license question it is the cleanest commercial "yes" among 2026's major releases. Boring licenses are a feature.

## Is DeepSeek open source?

Under the loose, license-based reading: yes. DeepSeek's recent flagship weights ship under the [MIT license](https://opensource.org/license/mit), per the [DeepSeek model cards](https://huggingface.co/deepseek-ai). MIT permits commercial use, modification, and redistribution with only a copyright-notice obligation. Under the strict OSAID: no, because the training data is not disclosed. For most builders the practical answer is the first one; just know which definition your legal team is using before you put "open source" in a slide deck.

## Can you use open models commercially? A 60-second checklist

Usually yes, but verify, don't assume. For every model in your stack:

1. **Open the license file on the model card.** Not the announcement post, not a news article. The license text is the only binding document.
2. **Classify it.** MIT or Apache 2.0 → proceed with standard obligations. Anything custom → read every clause.
3. **Scan for the four traps:** scale thresholds, geography carve-outs, naming/attribution duties, redistribution and derivative terms.
4. **Check the version history.** Licenses change between generations: Gemma went custom → Apache 2.0; DeepSeek's line went custom → MIT. Never inherit assumptions from a model's predecessor.
5. **Log the license in your model registry** alongside the evals. Our [model tracker](/models) lists license families for current releases if you need a starting point.
6. **Escalate to counsel** whenever the license is custom *and* you expect real scale, EU operations, or redistribution. That combination is where every trap lives.

## The license follows the weights

One last practitioner point: license terms are orthogonal to infrastructure. Whether you self-host, call the lab's own API, use a third-party serverless inference provider, or route through a decentralized inference marketplace such as Morpheus, the same license travels with the model. Running Llama 4 behind someone else's endpoint does not launder the EU carve-out, and running Kimi K3 on your own GPUs does not mute a naming clause. Choose your license posture and your infrastructure posture deliberately. They are separate decisions that both belong in writing.

## FAQ

**Is DeepSeek open source?**
DeepSeek's recent flagship weights ship under the OSI-approved MIT license, so commercial use, modification, and redistribution are permitted. Under the OSI's stricter Open Source AI Definition, which expects training-data transparency, it falls short. The data is not disclosed.

**What are the Llama 4 license's EU restrictions?**
Meta's Llama 4 Community License does not grant rights to the multimodal Llama 4 models to individuals domiciled in, or companies with a principal place of business in, the European Union. Text-only models remain available; EU teams deploying multimodal should get legal review.

**Kimi K3 license explained — what should you check?**
Kimi K3 ships under a custom Moonshot AI license rather than a standard OSI-approved text. Custom licenses can attach conditions to scale, naming, or redistribution, so read the full license on the model card before commercial deployment. The blog post is not the license.

**Can I use open models commercially?**
Usually yes. Models under Apache 2.0 or MIT (Gemma 4, DeepSeek) permit commercial use outright. Custom licenses (Llama 4, Kimi K3) permit it with conditions such as scale thresholds, attribution, or regional carve-outs. The binding answer is in the license text, not the launch post.

## FAQ

**Is DeepSeek open source?**

DeepSeek's recent flagship weights ship under the OSI-approved MIT license, so commercial use, modification, and redistribution are permitted. Under the OSI's stricter Open Source AI Definition, which expects training-data transparency, it falls short — the data is not disclosed.

**What are the Llama 4 license's EU restrictions?**

Meta's Llama 4 Community License does not grant rights to the multimodal Llama 4 models to individuals domiciled in, or companies with a principal place of business in, the European Union. Text-only models remain available; EU teams deploying multimodal should get legal review.

**Kimi K3 license explained — what should you check?**

Kimi K3 ships under a custom Moonshot AI license rather than a standard OSI-approved text. Custom licenses can attach conditions to scale, naming, or redistribution, so read the full license on the model card before commercial deployment — the blog post is not the license.

**Can I use open models commercially?**

Usually yes. Models under Apache 2.0 or MIT (Gemma 4, DeepSeek) permit commercial use outright. Custom licenses (Llama 4, Kimi K3) permit it with conditions such as scale thresholds, attribution, or regional carve-outs. The binding answer is in the license text, not the launch post.

## Sources

- [The Open Source AI Definition 1.0](https://opensource.org/ai/open-source-ai-definition) — Open Source Initiative
- [Llama 4 Community License Agreement](https://www.llama.com/llama4/license/) — Meta
- [Apache License, Version 2.0](https://www.apache.org/licenses/LICENSE-2.0) — Apache Software Foundation
- [The MIT License](https://opensource.org/license/mit) — Open Source Initiative
- [Moonshot AI model cards](https://huggingface.co/moonshotai) — Hugging Face
- [DeepSeek model cards](https://huggingface.co/deepseek-ai) — Hugging Face
- [Gemma models — Google AI for Developers](https://ai.google.dev/gemma) — Google


---

# What Is an OpenAI-Compatible API? Why It Kills Vendor Lock-In (2026)

*By DeAI Newsroom · 2026-08-24 · 7 min read*

Canonical: https://www.deai.org/news/openai-compatible-api

> **Summary:** An OpenAI-compatible API is any inference endpoint that implements OpenAI's request and response format, so existing OpenAI SDK code runs against it after a one-line base_url change.

An OpenAI-compatible API is any inference endpoint that speaks the OpenAI request and response format. Your existing OpenAI SDK code runs against it after a one-line change: set `base_url` to the new provider. That single line is what kills vendor lock-in.

## Key takeaways

- An OpenAI-compatible API reimplements OpenAI's HTTP interface (same endpoints, same JSON shapes), so client code keeps working with a 1-line `base_url` change.
- Compatibility is a spectrum: chat completions and streaming are near-universal; tool calling, structured outputs, and embeddings vary by provider.
- The ecosystem spans 4 categories: aggregators, self-hosted servers, hosted providers, and decentralized inference marketplaces, all reachable through the same SDK.
- The swap takes minutes, but verify streaming, error shapes, and model naming before routing production traffic.
- When switching costs approach zero, pricing and data-retention policy become competitive levers instead of traps.

## What is an OpenAI-compatible API?

An OpenAI-compatible API is an HTTP service that implements the same interface as OpenAI's API: you `POST` JSON to `/v1/chat/completions` with an `Authorization: Bearer` header and a body containing `model` and `messages`, and you get back a response whose answer lives at `choices[0].message.content`. If you ask for streaming, you get server-sent events with incremental `choices[0].delta` chunks, terminated by `data: [DONE]`.

The key point: "compatible" describes the **wire format**, not the company, the model, or the infrastructure behind it. Any provider (a GPU cloud, a local server on your laptop, a routing aggregator, a decentralized network) can implement that format. Once it does, every tool that already speaks OpenAI's dialect works with it: the official Python and Node SDKs, LangChain, LlamaIndex, most agent frameworks, and thousands of internal codebases written since 2023.

## Why did OpenAI's API become the de facto standard?

OpenAI shipped a simple chat-completions interface early, and the ecosystem standardized on it the way the web standardized on HTTP. Three forces locked it in:

1. **SDK ubiquity.** The official SDKs became the default client in tutorials, templates, and production code.
2. **Framework assumptions.** Orchestration libraries hard-coded OpenAI's request and response shapes as their canonical format.
3. **Supply-side adoption.** Open-source inference servers (vLLM, Ollama, llama.cpp) implemented the same endpoints because that is what client code already spoke. Every new provider followed, because "works with the OpenAI SDK" is the fastest way to reduce a customer's migration cost to zero.

The result is a rare situation in infrastructure: an interface owned by one vendor, implemented by everyone.

## How do you change the base URL in the OpenAI SDK?

This is the entire migration, mechanically speaking. Point the client at any OpenAI-compatible endpoint:

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://your-provider.example.com/v1",  # any OpenAI-compatible endpoint
    api_key="YOUR_PROVIDER_KEY",
)

response = client.chat.completions.create(
    model="your-model-name",
    messages=[{"role": "user", "content": "Hello"}],
)
print(response.choices[0].message.content)
```

The same swap with curl:

```bash
curl https://your-provider.example.com/v1/chat/completions \
  -H "Authorization: Bearer YOUR_PROVIDER_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "your-model-name",
    "messages": [{"role": "user", "content": "Hello"}]
  }'
```

Two details worth knowing. First, the SDK also respects the `OPENAI_BASE_URL` environment variable, so you can redirect an entire application (including tools that construct the client for you) without touching code. Second, the model string is provider-specific: each provider publishes its own model names, and that is the one value you will always update alongside the URL. For the full checklist (key management, model mapping, streaming tests, rollback), see our [guide to migrating from OpenAI](/migrate-from-openai).

## What has to match for a true drop-in OpenAI replacement?

"Drop-in" is a claim worth unpacking. For existing code to run unmodified, a provider needs to match on six things:

- **Endpoint paths.** At minimum `/v1/chat/completions`; ideally also `/v1/models` (so your code can enumerate what's available) and `/v1/embeddings` if you use them.
- **Authentication.** The `Authorization: Bearer ` header, nothing exotic.
- **Request and response schema.** Same field names, same nesting, same `usage` object for token counts.
- **Streaming format.** Server-sent events with `delta` chunks and the `[DONE]` sentinel, not a different chunking scheme that silently breaks your parser.
- **Error shapes.** OpenAI-style error bodies (`{"error": {"message", "type", "code"}}`), so your retry and alerting logic still fires correctly.
- **Tool calling.** The `tools` / `tool_calls` schema, if your application uses function calling.

Match all six and the swap is genuinely a config change. Match only the first three and simple chat works but your production paths may not.

## Where does compatibility break down?

The cracks appear at the edges of the API surface:

- **Newer OpenAI surfaces.** Many compatible providers implement chat completions and little else. The Responses API, realtime audio, assistants, and fine-tuning endpoints are far less commonly mirrored.
- **Structured outputs.** JSON-mode and schema-constrained generation are widely supported but differ in strictness: a schema one provider enforces, another may treat as a suggestion.
- **Sampling internals.** `logprobs`, `seed` determinism, and penalty parameters are implemented inconsistently across inference stacks.
- **Headers and limits.** Rate-limit headers, context-window sizes, and max-output defaults are provider-specific even when the body format is identical.
- **Model behavior.** The same open-weight model can be served with different quantization, chat templates, or safety layers, and outputs will differ. DeAI's refusal-index methodology scores how often served models decline benign prompts precisely because refusals can vary across providers serving identical weights, a behavior difference no API-format guarantee covers.

None of these are reasons to avoid compatible providers. They are reasons to test with your actual workload rather than assuming "compatible" means "identical."

## What OpenAI API alternatives are compatible with the SDK?

Compatible endpoints now come in four broad categories, all reachable with the same client code:

- **Aggregators and routers.** Services like OpenRouter front many models from many backends behind one key and one base URL. Useful when you want model choice without managing multiple accounts.
- **Self-hosted servers.** vLLM, Ollama, and llama.cpp's server mode all expose OpenAI-compatible endpoints on hardware you control. This is the maximum-control option: your weights, your logs, your retention.
- **Hosted open-weight providers.** A long list of GPU clouds and inference platforms serve open-weight models behind compatible endpoints, typically at a fraction of frontier-API pricing. Check each provider's published pricing page for current rates.
- **Decentralized inference marketplaces.** Morpheus is one example: a decentralized inference marketplace that routes requests to independent operators rather than a single company's datacenter. As with any provider, treat published privacy claims (zero-retention, operator non-visibility) as policy statements unless they have been independently audited.

To see which open-weight models are commonly served behind compatible endpoints, browse the [DeAI model catalog](/models).

## How do you verify compatibility before you commit?

A 30-minute checklist beats a week of surprises:

1. **Run your real prompts** through the candidate endpoint, not a "hello world."
2. **Test streaming explicitly**: confirm chunk boundaries and the `[DONE]` terminator work with your parser.
3. **Exercise tool calling and JSON mode** if you use them, including malformed-input cases.
4. **Trigger an error on purpose** (bad model name, oversized context) and inspect the error body.
5. **Read the retention policy.** Interface compatibility says nothing about what happens to your prompts after the response is sent.

## Why this kills vendor lock-in

Vendor lock-in is a switching-cost problem, and an OpenAI-compatible API collapses the biggest switching cost, rewriting client code, to one line. What remains is a config value, an API key, and a model name. That changes the economics of the whole market: you can run two providers in parallel and shift traffic gradually, fail over automatically when one degrades, route sensitive workloads to infrastructure you control and bulk workloads elsewhere, and negotiate from a position where leaving is cheap. Lock-in doesn't disappear entirely. It migrates to your evals, your prompt tuning, and your data. But the API itself stops being the cage.

## FAQ

### How do I change the base URL in the OpenAI SDK?

Pass `base_url` when constructing the client: `OpenAI(base_url="https://your-provider.example.com/v1", api_key="...")`. Every call then goes to that endpoint in the same format. With curl, swap `https://api.openai.com/v1` for the provider's URL. The `OPENAI_BASE_URL` env var works too.

### What is a drop-in OpenAI replacement?

A provider that implements the OpenAI API surface (`/v1/chat/completions`, bearer auth, streaming, and error shapes) closely enough that existing OpenAI SDK code runs against it with only a base-URL and API-key change, no rewrites.

### What OpenAI API alternatives are compatible with the SDK?

Several categories: aggregators such as OpenRouter, self-hosted servers like vLLM and Ollama, hosted open-weight providers, and decentralized inference marketplaces. Check each provider's docs for which endpoints (chat, embeddings, tools) they implement.

### Does OpenAI compatibility cover streaming and tool calling?

Usually yes for chat completions, but coverage varies. Streaming uses the same server-sent-event chunk format; tool calling and structured outputs are implemented by most major compatible providers, though edge cases differ. Verify against the provider's docs before migrating.

### Is an OpenAI-compatible API the same as the official OpenAI API?

No. It replicates the interface, not the service. Models, latency, uptime, data retention, and pricing are each provider's own. Compatibility means your client code doesn't change. Everything behind the endpoint can.

## FAQ

**How do I change the base URL in the OpenAI SDK?**

Pass base_url when constructing the client: OpenAI(base_url="https://your-provider.example.com/v1", api_key="..."). Every call then goes to that endpoint in the same format. With curl, swap https://api.openai.com/v1 for the provider's URL. The OPENAI_BASE_URL env var works too.

**What is a drop-in OpenAI replacement?**

A provider that implements the OpenAI API surface — /v1/chat/completions, bearer auth, streaming, and error shapes — closely enough that existing OpenAI SDK code runs against it with only a base-URL and API-key change, no rewrites.

**What OpenAI API alternatives are compatible with the SDK?**

Several categories: aggregators such as OpenRouter, self-hosted servers like vLLM and Ollama, hosted open-weight providers, and decentralized inference marketplaces. Check each provider's docs for which endpoints (chat, embeddings, tools) they implement.

**Does OpenAI compatibility cover streaming and tool calling?**

Usually yes for chat completions, but coverage varies. Streaming uses the same server-sent-event chunk format; tool calling and structured outputs are implemented by most major compatible providers, though edge cases differ. Verify against the provider's docs before migrating.

**Is an OpenAI-compatible API the same as the official OpenAI API?**

No. It replicates the interface, not the service. Models, latency, uptime, data retention, and pricing are each provider's own. Compatibility means your client code doesn't change — everything behind the endpoint can.

## Sources

- [OpenAI API Reference](https://platform.openai.com/docs/api-reference) — OpenAI
- [openai-python (official OpenAI SDK)](https://github.com/openai/openai-python) — OpenAI
- [OpenAI-Compatible Server](https://docs.vllm.ai/en/latest/serving/openai_compatible_server.html) — vLLM
- [Ollama OpenAI compatibility](https://github.com/ollama/ollama/blob/main/docs/openai.md) — Ollama
- [OpenRouter Documentation](https://openrouter.ai/docs) — OpenRouter
- [LiteLLM Documentation](https://docs.litellm.ai) — LiteLLM


---

# Top 9 OpenRouter Alternatives for Open Models (2026 — Priced)

*By DeAI Newsroom · 2026-08-24 · 7 min read*

Canonical: https://www.deai.org/news/openrouter-alternatives

> **Summary:** OpenRouter is a useful aggregator, but most teams serving open-weight models in production can go direct. These 9 providers all expose OpenAI-compatible endpoints, and switching is usually a one-line base-URL change.

The best OpenRouter alternatives in 2026 are the direct open-model inference providers: Together AI, Fireworks AI, DeepInfra, Groq, Cerebras, SambaNova, Nebius AI Studio, Hyperbolic, and the decentralized inference marketplace Morpheus — 9 options, all OpenAI-compatible, most reachable by changing one base URL.

## Key takeaways

- 9 providers serve open-weight models over OpenAI-compatible APIs, so switching from OpenRouter is typically a 2-line config change (base URL + API key).
- OpenRouter's docs describe a fee on credit purchases on top of pass-through provider pricing; going direct removes that layer, though list prices vary by model.
- 3 of the 9 (Groq, Cerebras, SambaNova) run custom silicon and position around latency; the rest run conventional GPU fleets.
- 1 of the 9 (Morpheus) is a decentralized inference marketplace, where independent operators rather than a single company serve the requests.
- Model ID strings differ across all 9: the single most common migration bug.

## What OpenRouter actually does (and why people look for alternatives)

OpenRouter is an aggregator: one API key, one bill, and a unified catalog that routes your request to an upstream provider hosting the model you asked for. That is genuinely useful for evaluation work, for apps that call many different models, and for automatic fallbacks when one upstream is down.

The trade-offs appear at production scale. You pay the aggregator layer (OpenRouter's docs describe a fee when you purchase credits, on top of pass-through upstream pricing), your rate limits are mediated by the aggregator rather than negotiated with the provider, and provider-specific features like dedicated deployments, batch endpoints, and fine-tuned model hosting often aren't exposed through the unified API. Teams that have settled on one or two production models usually end up asking the question this article answers: which provider to go direct to.

## How we picked the 9

Three criteria, applied evenly to every entry:

1. Open-weight model coverage: each provider serves a meaningful catalog of open-weight models (Llama, Qwen, DeepSeek, Mistral families and similar), not just its own fine-tunes.
2. OpenAI-compatible API: each exposes an endpoint you can point the standard OpenAI SDK at with a base-URL swap.
3. Self-serve access: you can get a key and make a call without a sales conversation.

Pricing below is described qualitatively because list prices change frequently. Every provider named publishes a pricing page, and DeAI tracks per-model provider pricing at [/prices](/prices); check both before committing.

## The 9 OpenRouter alternatives

### 1. Together AI

One of the largest dedicated open-model inference clouds. Together offers serverless per-token billing across a broad catalog, plus dedicated endpoints and reserved capacity for steady production traffic. It also hosts fine-tunes, which makes it a common landing spot for teams that train and serve in one place. Best for: teams that want one provider to cover both experimentation and production.

### 2. Fireworks AI

Fireworks competes on the same serverless-per-token model, with dedicated and on-demand GPU options on top. It has invested heavily in serving optimizations (quantized and speculative variants of popular models appear in its catalog) and offers batch pricing for offline workloads. Best for: latency-sensitive production apps that also have large offline batch jobs.

### 3. DeepInfra

DeepInfra built its reputation on aggressive list prices for popular open-weight models, billed per token on serverless infrastructure. The catalog skews toward the models people actually deploy (the major Llama, Qwen, and DeepSeek variants) rather than long-tail coverage. Best for: cost-driven teams running mainstream models at volume.

### 4. Groq

Groq runs its own LPU silicon instead of GPUs and positions the service around low-latency, high-throughput generation. The catalog is curated rather than broad (a rotating set of popular open-weight models), and it has historically offered a free tier useful for prototyping. Best for: interactive apps where tokens-per-second per user is the metric that matters.

### 5. Cerebras

Cerebras serves inference from its wafer-scale hardware and, like Groq, positions around generation speed on a curated set of flagship open-weight models. Per-token serverless billing, OpenAI-compatible endpoint. Best for: workloads dominated by long generation (agents, reasoning models, code) where wall-clock time compounds.

### 6. SambaNova

SambaNova is the third custom-silicon option (its RDU chips), with a cloud API that has historically included a free tier and an enterprise-facing pitch around running larger models at high throughput. Best for: enterprise buyers who want a vendor relationship alongside the API.

### 7. Nebius AI Studio

Nebius is a GPU cloud that also runs a serverless inference studio with per-token pricing on a solid open-weight catalog. The interesting angle is the upgrade path: you can start serverless and move to dedicated GPU capacity inside the same vendor as traffic grows. Best for: teams expecting to graduate from serverless to their own endpoints.

### 8. Hyperbolic

Hyperbolic combines a GPU rental marketplace with a serverless inference API, so it spans both "rent the card" and "pay per token" models. Its inference catalog covers the mainstream open-weight families at per-token pricing. Best for: teams that want one vendor for both bursty API traffic and raw GPU hours.

### 9. Morpheus

Morpheus is a decentralized inference marketplace: instead of one company operating the fleet, independent operators serve requests and the network routes between them, behind an OpenAI-compatible interface. The trust model is structurally different: no single operator sees all of your traffic by design. As with any provider's privacy claims, though, treat "operators can't see your prompts" style statements as policy, not verified fact. Pricing is set by the operator market rather than a single price list. Best for: builders who want inference without a single corporate counterparty.

Also worth a look: Hugging Face Inference Providers (itself an aggregator routing to several providers above), Baseten for dedicated deployments, Novita AI and Featherless for budget serverless, and Venice, which pairs a privacy-positioned consumer app with an API.

## OpenRouter vs direct API: what's the actual trade-off?

Stay on OpenRouter if you call many models across many providers, if automatic cross-provider fallback is worth a premium to you, or if one consolidated invoice materially simplifies your accounting.

Go direct if you have one or two production models, if you're hitting rate limits you can't negotiate through an intermediary, or if you need provider-specific features: dedicated endpoints, batch APIs, fine-tune hosting, committed-use discounts. Many teams do both: OpenRouter for the long tail and evals, a direct contract for the production workload.

## Is anything actually cheaper than OpenRouter?

Usually, on a like-for-like per-token basis. OpenRouter passes through upstream provider pricing and, per its own docs, adds a fee when you buy credits, so the same model on the same upstream provider typically costs list price when you buy direct. But "cheaper" is model-specific: providers price the same open-weight model differently, run different quantization, and discount differently at volume. The honest answer is to compare per-model prices across provider pricing pages (DeAI's tracker at [/prices](/prices) exists for exactly this comparison) rather than assume any single provider is universally cheapest.

## How do you switch from OpenRouter?

Mechanically, it's the easy part. Every provider above speaks the OpenAI chat-completions schema:

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://api.your-provider.com/v1",  # provider's endpoint
    api_key="YOUR_PROVIDER_KEY",
)

resp = client.chat.completions.create(
    model="provider-specific-model-id",  # check the provider's model list
    messages=[{"role": "user", "content": "Hello"}],
)
print(resp.choices[0].message.content)
```

```bash
curl https://api.your-provider.com/v1/chat/completions \
  -H "Authorization: Bearer $PROVIDER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "provider-specific-model-id", "messages": [{"role": "user", "content": "Hello"}]}'
```

The two real gotchas: model ID strings differ between providers (one's `llama-3.3-70b` is another's `meta-llama/Llama-3.3-70B-Instruct`), and response metadata like usage fields can vary slightly. For a full checklist (keys, model mapping, fallback logic, eval-before-cutover), see our [guide to switching from OpenRouter](/switch-from-openrouter).

## FAQ

**What are the best sites like OpenRouter?**
The strongest alternatives for open-weight models are Together AI, Fireworks AI, DeepInfra, Groq, Cerebras, SambaNova, Nebius AI Studio, Hyperbolic, and the decentralized inference marketplace Morpheus. All expose OpenAI-compatible endpoints.

**OpenRouter vs direct API: which should I use?**
OpenRouter when you want one key and one bill across many providers with fallbacks; direct when you have one or two production models and want list pricing, negotiable rate limits, and provider-specific features.

**Is anything cheaper than OpenRouter?**
Often, yes. OpenRouter passes through upstream pricing and charges a fee on credit purchases per its docs, so buying direct usually costs list price. Compare per-model prices before committing.

**How hard is it to switch?**
Usually a two-line change: swap the base URL and API key. The main gotcha is differing model ID strings between providers.

## FAQ

**What are the best sites like OpenRouter?**

The strongest OpenRouter alternatives for open-weight models are Together AI, Fireworks AI, DeepInfra, Groq, Cerebras, SambaNova, Nebius AI Studio, Hyperbolic, and the decentralized inference marketplace Morpheus. All expose OpenAI-compatible endpoints.

**OpenRouter vs direct API: which should I use?**

Use OpenRouter when you want one key and one bill across many providers with built-in fallbacks. Go direct when you have one or two production models and want list pricing, direct rate limits, and provider-specific features.

**Is anything cheaper than OpenRouter?**

Often, yes. OpenRouter passes through upstream provider pricing and, per its docs, charges a fee on credit purchases, so buying the same model direct from the provider usually costs list price. Compare per-model prices before committing.

**How hard is it to switch from OpenRouter to a direct provider?**

Usually a two-line change: swap the base URL and API key in your OpenAI-compatible client. The main gotcha is that model ID strings differ between providers, so check each provider's model list.

## Sources

- [OpenRouter Documentation](https://openrouter.ai/docs) — OpenRouter
- [Together AI — Inference Cloud](https://www.together.ai) — Together AI
- [Fireworks AI](https://fireworks.ai) — Fireworks AI
- [DeepInfra](https://deepinfra.com) — DeepInfra
- [GroqCloud](https://groq.com) — Groq
- [Cerebras Inference](https://www.cerebras.ai) — Cerebras
- [SambaNova Cloud](https://sambanova.ai) — SambaNova
- [Nebius AI Studio](https://nebius.com) — Nebius
- [Hyperbolic](https://www.hyperbolic.xyz) — Hyperbolic
- [Hugging Face Inference Providers](https://huggingface.co/docs/inference-providers) — Hugging Face


---

# OpenRouter vs Morpheus for Open Models (2026): Side by Side

*By DeAI Newsroom · 2026-08-24 · 8 min read*

Canonical: https://www.deai.org/news/openrouter-vs-morpheus

> **Summary:** OpenRouter is a centralized gateway to a broad model catalog with one API key; Morpheus is a decentralized inference marketplace where independent providers serve open-weight models.

OpenRouter vs Morpheus comes down to one decision: who you want standing between your prompts and the GPU. OpenRouter is a single managed gateway to a broad model catalog; Morpheus is a decentralized inference marketplace where independent providers serve open-weight models. Neither is categorically cheaper, faster, or more private.

## Key takeaways

- Both expose **OpenAI-compatible endpoints**. Moving a workload between them is a 2-line change: the base URL and the API key.
- **OpenRouter** is 1 company operating a managed gateway over a broad catalog (proprietary and open-weight). **Morpheus** is a decentralized inference marketplace of independent providers serving open-weight models.
- On privacy, 0 of the 2 providers' no-logging postures are independently verified as of 2026-08-20. Treat both as policy statements and pick the trust model you prefer.
- Neither is categorically cheaper; both are typically a fraction of frontier-API pricing for open-weight models. Confirm on each provider's published pricing page.
- Refusals and behavior are model properties, not router properties. DeAI's refusal-index methodology scores them per model. See the [tracker](/tracker).

*Disclosure: DeAI News is an independent publication covering decentralized AI infrastructure, the category in which Morpheus operates. Neither provider sponsored, reviewed, or influenced this comparison; both were assessed against identical criteria.*

## What are OpenRouter and Morpheus, exactly?

**OpenRouter** is a hosted model-routing gateway. You create one account, get one API key, and call an OpenAI-compatible endpoint that fronts a large catalog: proprietary frontier models and open-weight releases alike. OpenRouter handles authentication, metering, and routing across the upstream inference providers actually running the GPUs, and it documents fallback behavior when one upstream is unavailable.

**Morpheus** is a decentralized inference marketplace. Independent compute providers list capacity, and consumers reach open-weight models through an OpenAI-compatible interface. Instead of a conventional SaaS account, access is mediated by the network's token mechanism (staking for a share of inference capacity), and settlement happens through the marketplace. The project's stated premise is that no single company should sit in the middle of every request.

Same API shape, opposite plumbing. That is the whole comparison in one sentence.

## OpenRouter vs Morpheus: side by side on the same criteria

| Criterion | OpenRouter | Morpheus | Edge |
|---|---|---|---|
| Trust model | One operator's gateway plus its upstream providers | Distributed across independent providers you route among | Tie: pick your trust model |
| Catalog breadth | Proprietary frontier + open-weight models | Open-weight models; depth varies with providers online | OpenRouter |
| Architectural privacy | All traffic transits a single gateway | No single operator in the middle (project's design claim) | Morpheus (claimed, unverified) |
| Verified privacy | Policy statements only | Policy statements only | Tie |
| Permissionlessness | Operator controls listings and can delist | Permissionless provider entry (project's description) | Morpheus |
| Billing & procurement | Conventional account, per-token billing | Token-mediated access plus marketplace pricing | OpenRouter |
| API compatibility | OpenAI-compatible | OpenAI-compatible | Tie |
| Failover | Documented cross-provider fallback | Redundancy via multiple independent providers | Tie: different mechanisms |
| Cost | Per-token, listed per model | Set by marketplace providers | Tie: check pricing pages |
| Ecosystem maturity | Longer-running, broader docs and integrations | Younger, smaller tooling surface | OpenRouter |

The table nets out to three OpenRouter edges, two Morpheus edges, and five ties, which is the honest picture. Neither wins outright, and anyone telling you otherwise is selling something. The rows that actually decide it for most teams are the first one (trust model) and the sixth (procurement).

## OpenRouter privacy: who can see your prompts?

Trace the data path. Your request goes from your client to OpenRouter's gateway, and OpenRouter forwards it to the upstream provider serving that model. That means at least two parties' policies apply to every prompt: OpenRouter's and the upstream provider's.

OpenRouter publishes a privacy policy and documents routing preferences that let you constrain which upstream providers can serve your traffic. That is genuinely useful, but it is still a policy framework, not a proof. Upstream providers have their own data-handling terms, which vary, and OpenRouter's policy is a statement about what OpenRouter says it does, not an independently verified fact. As of 2026-08-20, neither provider in this comparison points to a public third-party audit verifying its logging claims.

Practical guidance: if a prompt would hurt you in a breach or a subpoena, assume any hosted router can see it unless you have a contractual guarantee saying otherwise. For stricter threat models, the options are self-hosting open weights or choosing providers whose specific terms you have reviewed, rather than trusting marketing pages on either side.

## Morpheus API review: what is it like to build on?

From the client's perspective, the Morpheus API is familiar: an OpenAI-compatible chat-completions endpoint that works with the standard SDKs. What differs is everything behind the base URL.

Access is mediated by the network's staking mechanism rather than a credit card and a dashboard. That is a real friction difference. Some teams will find token-mediated access a non-starter for procurement reasons; others will accept it as the cost of the trust properties they want. Both reactions are reasonable.

The catalog is open-weight models, and depth at any moment depends on which providers are online and what they are serving. The ecosystem is younger than OpenRouter's, with a smaller surface of managed conveniences: usage dashboards, team management, invoicing, and long-tail integrations. Morpheus says its peer-routed design means no single operator can observe all traffic; like OpenRouter's policy, that is a design and policy statement, not an independently verified fact.

Verdict on the API itself: workable today for builders comfortable with a younger ecosystem, and worth piloting on non-critical workloads before you route production traffic through it.

## Which is cheaper, OpenRouter or Morpheus?

There is no verified basis to call either one cheaper, and this comparison won't invent numbers to fill the gap. OpenRouter lists per-token prices per model on its site, sometimes with several price points for the same model from different upstream providers. Morpheus pricing is set by the marketplace's providers. For comparable open-weight models, both are typically a fraction of frontier-API pricing. The only reliable move is to price the exact model you plan to run on each provider's published pricing page.

## How hard is it to switch between them?

Trivially easy, because both speak the OpenAI chat-completions schema. The swap is two lines (base URL and API key), and each provider publishes its own base URL in its docs.

Python:

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://provider-endpoint.example/v1",  # swap per provider
    api_key="YOUR_API_KEY",
)

resp = client.chat.completions.create(
    model="open-weight-model-name",  # check each provider's catalog
    messages=[{"role": "user", "content": "Hello"}],
)
print(resp.choices[0].message.content)
```

curl:

```bash
curl https://provider-endpoint.example/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "open-weight-model-name", "messages": [{"role": "user", "content": "Hello"}]}'
```

Two caveats: model naming conventions differ between catalogs, so confirm the exact identifier; and behavior differences you observe after switching are almost always the model or the provider's serving config, not the routing layer. DeAI's refusal-index methodology scores refusal behavior per model for exactly this reason, and the [tracker](/tracker) follows how served models change over time.

## What about reliability and failover?

OpenRouter documents automatic fallback across upstream providers when one is unavailable, which is a real operational convenience. Morpheus gets redundancy a different way: multiple independent providers can list the same model, so capacity is not tied to one operator's uptime. Neither should be assumed to carry an SLA unless your specific agreement says so. For either option, implement client-side retries with backoff. For truly critical workloads, dual-route across both, since the client code is already identical.

## Which one should you choose?

Choose **OpenRouter** if you want one bill and one API key, need proprietary frontier models alongside open weights, or have a conventional procurement process that expects invoices and account management.

Choose **Morpheus** if your priority is a trust model with no single operator in the middle, you are committed to open-weight models anyway, and you can live with token-mediated access and a younger tooling ecosystem.

Or run both. Because the endpoints are OpenAI-compatible, plenty of teams route per workload: sensitive traffic according to their own threat model, bulk traffic wherever the price-performance is best that month. If neither fits, the DeAI News roundup of [OpenRouter alternatives](/openrouter-alternatives) covers the wider field, including self-hosting paths.

## FAQ

### OpenRouter vs Morpheus: which is better for open models?

Neither wins outright. OpenRouter offers a broader catalog and conventional billing; Morpheus offers a decentralized trust model via a marketplace of independent providers. Both expose OpenAI-compatible endpoints, so you can run both and route per workload.

### Morpheus API review: is it production-ready?

Morpheus exposes an OpenAI-compatible endpoint backed by a decentralized inference marketplace. It works with standard OpenAI SDKs, but the ecosystem is younger and catalog depth varies with which providers are online. Pilot on non-critical workloads first.

### OpenRouter privacy: does OpenRouter see my prompts?

Prompts transit OpenRouter's gateway and are forwarded to the upstream provider serving the model, so both parties' policies apply. OpenRouter publishes a privacy policy, but like all such documents it is a policy statement, not an independently verified fact.

### Is Morpheus cheaper than OpenRouter?

There is no verified basis to call either cheaper. Both are typically a fraction of frontier-API pricing, and prices vary by model and provider. Compare the exact model you need on each provider's published pricing page.

### Can I use the same code with both OpenRouter and Morpheus?

Yes. Both expose OpenAI-compatible chat-completions endpoints. Switching is a two-line change with the standard OpenAI SDK: set the provider's base URL and API key. Model names may differ, so check each catalog.

## FAQ

**OpenRouter vs Morpheus: which is better for open models?**

Neither wins outright. OpenRouter offers a broader catalog and conventional billing; Morpheus offers a decentralized trust model via a marketplace of independent providers. Both expose OpenAI-compatible endpoints, so you can run both and route per workload.

**Morpheus API review: is it production-ready?**

Morpheus exposes an OpenAI-compatible endpoint backed by a decentralized inference marketplace. It works with standard OpenAI SDKs, but the ecosystem is younger and catalog depth varies with which providers are online. Pilot on non-critical workloads first.

**OpenRouter privacy: does OpenRouter see my prompts?**

Prompts transit OpenRouter's gateway and are forwarded to the upstream provider serving the model, so both parties' policies apply. OpenRouter publishes a privacy policy, but like all such documents it is a policy statement, not an independently verified fact.

**Is Morpheus cheaper than OpenRouter?**

There is no verified basis to call either cheaper. Both are typically a fraction of frontier-API pricing, and prices vary by model and provider. Compare the exact model you need on each provider's published pricing page.

**Can I use the same code with both OpenRouter and Morpheus?**

Yes. Both expose OpenAI-compatible chat-completions endpoints. Switching is a two-line change with the standard OpenAI SDK: set the provider's base URL and API key. Model names may differ, so check each catalog.

## Sources

- [OpenRouter Documentation](https://openrouter.ai/docs) — OpenRouter
- [OpenRouter Privacy Policy](https://openrouter.ai/privacy) — OpenRouter
- [OpenRouter Models](https://openrouter.ai/models) — OpenRouter
- [Morpheus](https://mor.org) — Morpheus
- [OpenAI API Reference](https://platform.openai.com/docs/api-reference) — OpenAI


---

# How to Run DeepSeek V4 Flash in 2026: The $0.14/M Workhorse

*By DeAI Newsroom · 2026-08-24 · 9 min read*

Canonical: https://www.deai.org/news/run-deepseek-v4-flash-api

> **Summary:** DeepSeek V4 Flash is the budget tier of DeepSeek's V4 open-weight family, listed at $0.14 per million input tokens and $0.28 per million output as of 2026-08-20.

DeepSeek V4 Flash is the budget tier of DeepSeek's V4 open-weight line, listed at $0.14 per million input tokens and $0.28 per million output tokens on the official API as of 2026-08-20. This guide covers where to run it, how to switch with a two-line base-URL change, and how cache-hit pricing can shrink that bill further.

## Key takeaways

- V4 Flash lists at **$0.14/M input and $0.28/M output** on DeepSeek's official API (as of 2026-08-20), typically a fraction of frontier-API pricing.
- A workload of **5M input + 1M output tokens per day costs about $0.98/day (~$29/month)** at list price, before any cache discounts.
- Switching from any OpenAI-compatible provider is a **two-line change**: `base_url` and `api_key`, plus the model string.
- **Cache-hit pricing** bills repeated prompt prefixes at a discounted rate instead of full input price, and on the official API it's automatic with no opt-in required.
- Hosting options span the official API, aggregators, inference clouds, and decentralized marketplaces; compare them on price, cache-hit passthrough, rate limits, and retention terms.

## What is DeepSeek V4 Flash?

V4 Flash is the lighter, cost-optimized member of DeepSeek's V4 open-weight family. Where the V4 Pro tier targets maximum capability on hard reasoning and long-horizon tasks, Flash is built for the high-volume work that dominates most production traffic: summarization, classification, extraction, routing, chat, and retrieval-augmented generation.

Because the weights are published under DeepSeek's Hugging Face organization, you are not locked to one vendor. The same model can be served by DeepSeek's own API, third-party inference clouds, aggregators, or your own hardware, which is why a base-URL-swap workflow matters. For exact specs (context window, license terms, tokenizer details), check the model card on Hugging Face rather than any blog post, including this one.

## DeepSeek V4 Flash vs V4 Pro: which should you run?

The structural difference is simple: Flash is the cost-optimized tier, Pro is the capability tier. In practice, that suggests a routing pattern rather than an either/or decision:

- **Default everything to Flash.** Most production prompts (extraction, rewriting, classification, short-form chat, RAG answers with grounded context) are well within a workhorse model's range.
- **Escalate failures to Pro.** Add a lightweight check (confidence heuristics, output validation, user feedback signals) and re-run the small share of prompts that fail on Flash against V4 Pro.
- **Keep both behind one client.** Since both tiers are served through OpenAI-compatible endpoints, tier routing is a model-string change, not a re-integration.

If your workload is mostly difficult reasoning, start with Pro instead; see our companion guide, [How to Run DeepSeek V4 Pro](/run-deepseek-v4-pro-api). For everything else, Flash-first routing is where the cost savings come from.

## Where is the cheapest place to host DeepSeek V4 Flash?

There is no single honest answer, because "cheapest" depends on your traffic shape. The verified anchor is the official DeepSeek API list price: **$0.14/M input, $0.28/M output, as of 2026-08-20**. Every other host sets its own rates, and those rates change, so treat any specific third-party number you see in a blog post as stale until you confirm it on the provider's pricing page.

Your realistic options, on identical criteria:

| Option type | Examples | How pricing works | What to verify |
|---|---|---|---|
| Official API | DeepSeek API | List price, with automatic cache-hit discount | Rate limits, regional availability |
| Aggregator | OpenRouter | Routes to multiple upstreams; prices vary per upstream | Which upstream serves you, cache-hit passthrough |
| Inference cloud | Together, Fireworks, Novita | Provider-set per-token rates | Cache billing, dedicated vs shared capacity |
| Decentralized marketplace | Morpheus, a decentralized inference marketplace | Independent operators set their own rates | Per-operator pricing, retention policies |

Two cautions apply across the board. First, a lower sticker price can be erased if the provider doesn't pass through cache-hit billing: a workload with 80% cached prefixes can cost less at a higher list price with cache discounts than at a lower flat rate. Second, data-retention terms differ by provider, and any "zero-retention" style language is a policy statement by that provider, not an independently verified fact. Read the terms; don't take marketing pages at face value.

For a regularly updated comparison of per-token prices across hosts, see [our cheapest LLM API roundup](/cheapest-llm-api).

## How do you switch your code to V4 Flash?

If you already use the OpenAI SDK (or any OpenAI-compatible client), migration is a configuration change, not a rewrite. Keep the endpoint and key in environment variables so you can move providers without touching code:

```python
import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["PROVIDER_API_KEY"],
    base_url=os.environ["PROVIDER_BASE_URL"],  # e.g. https://api.deepseek.com
)

resp = client.chat.completions.create(
    model="deepseek-v4-flash",  # exact model string varies by provider
    messages=[
        {"role": "system", "content": "You are a precise extraction engine."},
        {"role": "user", "content": "Extract the invoice total from: ..."},
    ],
    temperature=0.2,
)

print(resp.choices[0].message.content)
print(resp.usage)  # inspect cache hit/miss fields where supported
```

The same call in curl:

```bash
curl "$PROVIDER_BASE_URL/chat/completions" \
  -H "Authorization: Bearer $PROVIDER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-v4-flash",
    "messages": [
      {"role": "system", "content": "You are a precise extraction engine."},
      {"role": "user", "content": "Extract the invoice total from: ..."}
    ],
    "temperature": 0.2
  }'
```

Base URLs you'll commonly encounter include `https://api.deepseek.com` for the official API, `https://openrouter.ai/api/v1` for OpenRouter, `https://api.together.xyz/v1` for Together, and `https://api.fireworks.ai/inference/v1` for Fireworks. The exact model string (`deepseek-v4-flash` versus a vendor-prefixed variant) differs per provider, so check each provider's docs for the current identifier before deploying.

## How does cache-hit pricing actually work?

This is the most underused lever in LLM cost control, and it's worth understanding precisely.

**The mechanism.** When you send a prompt, the provider checks whether the *beginning* of your token sequence matches a prefix it processed recently. Tokens that match a cached prefix are billed at a discounted **cache-hit rate**; only the novel suffix is billed at the full input rate (the "cache miss" rate). On DeepSeek's official API this is automatic; there is nothing to enable and no API parameter to set. The official pricing page lists the current cache-hit and cache-miss input rates separately; check it for the live numbers, since only the $0.14/M list input price is verified as of 2026-08-20.

**Why it matters.** Many real workloads resend nearly identical prefixes on every call:

- A long system prompt with instructions and tool definitions
- Few-shot examples prepended to every request
- The same document or knowledge base chunk used across many RAG queries
- Agent loops that resend the full conversation history each turn

In these patterns, the majority of input tokens can qualify as cache hits, which pulls your *effective* input rate well below list. The formula is:

```
effective input rate = (miss_tokens × input_rate + hit_tokens × hit_rate) / total_input_tokens
```

**How to maximize hits.** Prefix matching works from the start of the prompt forward, so structure accordingly:

1. **Stable content first, variable content last.** System prompt, tool schemas, and reference documents go at the top; the user's actual question goes at the end.
2. **Never put entropy in the prefix.** Timestamps, request IDs, and random session tokens near the top of the prompt break the match for everything after them.
3. **Batch similar requests together.** Cache windows are time-bound, so grouping requests that share a prefix improves hit rates.
4. **Keep few-shot examples byte-identical.** Even small edits to examples reset the cached region to the edit point.

**How to verify you're getting hits.** DeepSeek's official API reports `prompt_cache_hit_tokens` and `prompt_cache_miss_tokens` in the response's `usage` object. Log both per request and compute your hit ratio weekly; if it's low, your prompt structure is the problem, not the pricing. Field names differ across providers, and some third-party hosts don't break out cache statistics at all, which is itself a reason to prefer hosts that do.

**One caveat:** not every host passes cache-hit billing through to customers. Some charge a flat input rate regardless. Before committing to a provider, confirm on its pricing page whether cached tokens are billed separately.

## What will V4 Flash cost per month?

Using only the verified list prices ($0.14/M input, $0.28/M output, as of 2026-08-20), and assuming zero cache hits as a conservative floor:

| Daily volume | Daily cost | 30-day cost |
|---|---|---|
| 5M input + 1M output | $0.98 | ~$29 |
| 50M input + 10M output | $9.80 | ~$294 |

Every percentage point of cache hits lowers the input side of that table. A workload with a heavy stable prefix (say, a support bot with a large system prompt) can see its effective input cost drop substantially below the figures above. Run your own prompt mix for a week, read the cache hit/miss fields, and do the arithmetic with the formula in the previous section.

## Production checklist

Before you point real traffic at V4 Flash:

- **Pin the model version** if your provider offers dated snapshots, so silent upgrades don't shift output behavior.
- **Set `max_tokens` explicitly** on every call; output tokens are billed at 2× the input rate and runaway generations are the most common budget surprise.
- **Implement retry-with-backoff** for rate limits and 5xx responses, and cap retries to avoid duplicate billing loops.
- **Log the full `usage` object**, including cache hit/miss fields, so you can audit effective pricing against the provider's invoice.
- **Keep a fallback provider configured.** Because switching is a base-URL change, a secondary endpoint costs nothing until you need it.
- **Test structured output and tool calling** on your chosen host specifically; support for these features can vary across providers serving the same weights.
- **Read the data-retention terms** of whichever host you pick, and treat retention claims as policy statements rather than verified guarantees.

## FAQ

### What is the difference between DeepSeek V4 Flash and V4 Pro?

Flash is the cost-optimized tier of DeepSeek's V4 open-weight family, listed at $0.14/M input tokens as of 2026-08-20; Pro is the higher-capability tier at a higher price. Default to Flash for high-volume tasks and escalate hard prompts to Pro.

### What is the cheapest way to run DeepSeek V4 Flash?

The official API lists $0.14/M input and $0.28/M output as of 2026-08-20. Third-party hosts and decentralized marketplaces set their own rates, so compare per-token price, cache-hit support, and rate limits on each provider's pricing page.

### How does DeepSeek cache-hit pricing work?

When the start of your prompt matches a recently processed prefix, those tokens are billed at a discounted cache-hit rate instead of the full input rate. It's automatic on the official API; put stable content first and variable content last to maximize hits.

### Is DeepSeek V4 Flash OpenAI-compatible?

Yes. The official API and most third-party hosts expose an OpenAI-compatible chat-completions endpoint, so switching is usually a matter of changing `base_url`, `api_key`, and the model name in your existing client code.

## FAQ

**What is the difference between DeepSeek V4 Flash and V4 Pro?**

Flash is the cost-optimized tier of DeepSeek's V4 open-weight family, listed at $0.14/M input tokens as of 2026-08-20; Pro is the higher-capability tier at a higher price. Default to Flash for high-volume tasks and escalate hard prompts to Pro.

**What is the cheapest way to run DeepSeek V4 Flash?**

The official API lists $0.14/M input and $0.28/M output as of 2026-08-20. Third-party hosts and decentralized marketplaces set their own rates, so compare per-token price, cache-hit support, and rate limits on each provider's pricing page.

**How does DeepSeek cache-hit pricing work?**

When the start of your prompt matches a recently processed prefix, those tokens are billed at a discounted cache-hit rate instead of the full input rate. It's automatic on the official API; put stable content first and variable content last to maximize hits.

**Is DeepSeek V4 Flash OpenAI-compatible?**

Yes. The official API and most third-party hosts expose an OpenAI-compatible chat-completions endpoint, so switching is usually a matter of changing base_url, api_key, and the model name in your existing client code.

## Sources

- [DeepSeek API Docs](https://api-docs.deepseek.com/) — DeepSeek
- [DeepSeek API Pricing](https://api-docs.deepseek.com/quick_start/pricing) — DeepSeek
- [deepseek-ai on Hugging Face](https://huggingface.co/deepseek-ai) — Hugging Face
- [OpenRouter Documentation](https://openrouter.ai/docs) — OpenRouter
- [Together AI Documentation](https://docs.together.ai/) — Together AI
- [Fireworks AI Documentation](https://docs.fireworks.ai/) — Fireworks AI
- [Novita AI](https://novita.ai/) — Novita AI


---

# How to Run DeepSeek V4 Pro via API in 2026 — Every Host Compared

*By DeAI Newsroom · 2026-08-24 · 8 min read*

Canonical: https://www.deai.org/news/run-deepseek-v4-pro-api

> **Summary:** DeepSeek V4 Pro is reachable through the official DeepSeek API, third-party OpenAI-compatible hosts, aggregators, decentralized marketplaces, or your own GPUs. DeepSeek self-reports 80.6% on SWE-bench Verified as of 2026-08-20 — a vendor claim until independent evals land.

You can run DeepSeek V4 Pro through the official DeepSeek API, third-party OpenAI-compatible hosts, aggregators, decentralized marketplaces, or your own hardware. DeepSeek reports 80.6% on SWE-bench Verified (self-reported, as of 2026-08-20). This guide compares every hosting route on jurisdiction, pricing model, and setup effort, with copy-paste code for each.

## Key takeaways

- DeepSeek self-reports **80.6% on SWE-bench Verified** for V4 Pro (as of 2026-08-20) — a vendor claim, not an independent result.
- Five hosting routes exist: official API, third-party serverless hosts, aggregators, decentralized marketplaces, and self-hosting. Nearly all expose an OpenAI-compatible endpoint, so switching is a two-line change.
- The official API is operated from China, and DeepSeek's privacy policy states user data is stored on servers there; third-party hosts run the same open weights on their own infrastructure.
- No prices are compared in this guide because every host sets its own per-token rates; verify on each provider's pricing page before committing.
- SWE-bench Verified is a 500-task benchmark; one score never captures latency, tool-use reliability, or refusal behavior.

## What is DeepSeek V4 Pro?

DeepSeek V4 Pro is the flagship open-weight release in DeepSeek's V4 line, aimed primarily at agentic coding and tool-use workloads. The headline number: DeepSeek reports **80.6% on SWE-bench Verified**, self-reported as of 2026-08-20. SWE-bench Verified is a 500-task, human-validated benchmark built from real GitHub issues, so if the figure holds up under independent reproduction, it places V4 Pro in frontier coding territory.

Treat it as a vendor claim for now. Third-party evaluators typically need weeks to replicate scaffolding, prompts, and sandbox setup before confirming or discounting a self-reported score. For side-by-side context against other open-weight releases, see the [model directory](/models). Separately, DeAI's refusal-index methodology scores how often hosted models decline benign requests; V4 Pro entries will appear as runs complete.

## Where can you run the DeepSeek V4 API?

Because the weights are open, "the DeepSeek V4 API" is not one endpoint. There are five distinct routes, each with different jurisdiction, billing, and data-handling properties.

| Route | Examples | Jurisdiction | Pricing model | Setup effort |
|---|---|---|---|---|
| Official DeepSeek API | api.deepseek.com | China; DeepSeek's privacy policy states data is stored on servers there | Per-token, published on DeepSeek's docs | Lowest |
| Third-party serverless hosts | Together AI, Fireworks AI, Novita AI, Hyperbolic, Cerebras, SambaNova | US / EU / varies by provider | Per-token (some per-GPU-second); each publishes a pricing page | Low |
| Aggregators | OpenRouter | Depends on the upstream you select | Per-token pass-through plus a margin | Low |
| Decentralized marketplaces | Morpheus | Varies by operator | Market-set per-token | Low–medium |
| Self-hosting | Your GPUs or rented bare metal | Yours | GPU-hours only | High |

### The official DeepSeek API

The first-party endpoint at `api.deepseek.com` is OpenAI-compatible and historically the first place new DeepSeek features (context caching, off-peak discount windows) appear. The exact V4 Pro model slug, rate limits, and current pricing live on DeepSeek's API docs. The trade-offs are jurisdiction (covered below) and, for some teams, procurement and payment friction from certain regions. If neither affects you, it is the simplest path to the model.

### Third-party hosted providers

Because V4 Pro ships as open weights, US- and EU-based inference providers can serve it on their own infrastructure. Together AI, Fireworks AI, Novita AI, Hyperbolic, Cerebras, and SambaNova have all hosted DeepSeek open-weight releases; whether each carries the exact V4 Pro checkpoint changes week to week, so check their current model lists. Most expose OpenAI-compatible endpoints and publish per-token pricing pages; some also sell dedicated throughput for production workloads. Data handling is governed by each provider's retention policy, and any "zero-retention" language is a policy statement, not an independently verified fact.

### Aggregators

OpenRouter sits in front of multiple upstreams: one API key, one bill, and the ability to pin or exclude providers by region. That makes it the fastest way to compare hosts serving identical weights, at the cost of a per-token margin and an extra party in the request path. If you use an aggregator for failover, read its data-policy page: your prompts transit the aggregator before reaching the upstream provider.

### Decentralized marketplaces

Morpheus, a decentralized inference marketplace, routes requests to independent operators rather than a single company's data centers. Pricing is market-set, and jurisdiction depends on which operator actually serves your request, a property to evaluate per workload, not assume. As with every host in this guide, retention and privacy terms are claims made by the operator or marketplace, not facts DeAI has verified. The category's practical appeal is permissionless access and provider diversity; the practical cost is that guarantees vary operator by operator.

### Self-hosting

Weights are distributed through DeepSeek's Hugging Face organization, where the model card carries license terms and hardware guidance. Serve them with vLLM or SGLang. Both expose an OpenAI-compatible server, so downstream code looks identical to any hosted option. Self-hosting converts per-token spend into GPU-hours and gives you full control over logs and retention, at the price of owning capacity planning, quantization trade-offs, and uptime. For steady, high-volume workloads it often pencils out; for bursty traffic, hosted is simpler.

## How do you call the DeepSeek V4 API?

Every hosted route above speaks the OpenAI chat-completions schema, so integration is a three-line change: base URL, API key, model slug. Model slugs differ per host, so copy the exact string from your provider's docs.

Python, using the official OpenAI SDK:

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://api.deepseek.com",  # swap for your host's endpoint
    api_key="YOUR_API_KEY",
)

response = client.chat.completions.create(
    model="deepseek-v4-pro",  # use your host's exact model slug
    messages=[{"role": "user", "content": "Refactor this function for O(n) time."}],
    temperature=0.2,
)

print(response.choices[0].message.content)
```

The same call with curl:

```bash
curl https://api.deepseek.com/chat/completions \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-v4-pro",
    "messages": [{"role": "user", "content": "Refactor this function for O(n) time."}]
  }'
```

For self-hosting, point `base_url` at `http://localhost:8000/v1` once your vLLM or SGLang server is up; the request body is unchanged.

## Can you use DeepSeek V4 without the China-based official API?

Yes. That is the practical consequence of open weights. DeepSeek's own privacy policy states that user data is stored on servers in China; that is a policy statement from the vendor, and it is the fact most compliance teams anchor on. If your organization answers to GDPR, data-residency clauses, or customer contracts that restrict where prompts are processed, the official API may fail review regardless of model quality.

The alternatives keep the weights and change the operator: third-party hosts in the US or EU, aggregators pinned to specific regions, decentralized marketplaces (where operator jurisdiction varies), or self-hosting in your own cloud. One nuance worth stating plainly: routing around the official API changes who processes your data, not who built the model. For most regulatory regimes, inference location and prompt handling are what matter, but confirm that with your counsel, not with a blog post. DeAI breaks these trade-offs down in detail in [DeepSeek API: official vs. third-party](/deepseek-api-official-vs-third-party).

## How much does the DeepSeek V4 Pro API cost?

There is no single price: every host sets its own per-token rates, and DeAI does not republish figures that change weekly. Three reliable rules of thumb. First, open-weight models typically run at a fraction of frontier closed-API pricing, regardless of host. Second, the official API has historically posted very low list prices relative to closed frontier APIs, with context-caching and off-peak mechanics described on DeepSeek's docs; verify current numbers there. Third, aggregators add a margin on upstream rates, while self-hosting converts the question into GPU-hours.

The only comparison that matters is on your own token mix: input-to-output ratio, cache hit rate, and peak concurrency. Pull each provider's pricing page (linked in the sources) and run your real traffic profile against it.

## How to choose a host: five checks

1. **Jurisdiction first.** If the official API fails compliance review, shortlist only hosts in acceptable regions. This single filter usually halves the field.
2. **Confirm the exact checkpoint.** "DeepSeek V4" listings can mean different variants; verify the provider serves V4 Pro, not a smaller sibling.
3. **Price your real traffic.** Input-heavy RAG and output-heavy codegen bill very differently; model your input:output ratio against each pricing page.
4. **Test rate limits before migrating.** Per-minute token caps and burst behavior differ widely across hosts, and they are rarely on the marketing page.
5. **Treat privacy terms as policy statements.** Zero-retention and "operators can't see prompts" language is a claim to weigh in your risk review, not a verified property.

## FAQ

### How much does the DeepSeek V4 Pro API cost?

Pricing is per token and set by each host. DeepSeek publishes its rates on its API docs; third-party hosts and aggregators publish their own pricing pages. Open-weight models typically run at a fraction of frontier closed-API pricing, so compare on your real input:output mix.

### Can I use DeepSeek V4 without the China-based official API?

Yes. Because V4 Pro ships as open weights, third-party hosts in the US and EU, aggregators, decentralized marketplaces, and self-hosting all serve the model without routing prompts through DeepSeek's official API. Jurisdiction then depends on the operator you choose.

### What is DeepSeek V4 Pro's SWE-bench score?

DeepSeek reports 80.6% on SWE-bench Verified, self-reported as of 2026-08-20. SWE-bench Verified is a 500-task, human-validated coding benchmark. Treat the figure as a vendor claim until independent evaluations reproduce it.

### Is DeepSeek V4 Pro open-weight?

Yes. Weights are distributed via DeepSeek's Hugging Face organization, which is what makes third-party hosting and self-hosting possible. Check the model card for license terms and hardware requirements before deploying.

## FAQ

**How much does the DeepSeek V4 Pro API cost?**

Pricing is per token and set by each host. DeepSeek publishes its rates on its API docs; third-party hosts and aggregators publish their own pricing pages. Open-weight models typically run at a fraction of frontier closed-API pricing — compare on your real input:output mix.

**Can I use DeepSeek V4 without the China-based official API?**

Yes. Because V4 Pro ships as open weights, third-party hosts in the US and EU, aggregators, decentralized marketplaces, and self-hosting all serve the model without routing prompts through DeepSeek's official API. Jurisdiction then depends on the operator you choose.

**What is DeepSeek V4 Pro's SWE-bench score?**

DeepSeek reports 80.6% on SWE-bench Verified, self-reported as of 2026-08-20. SWE-bench Verified is a 500-task, human-validated coding benchmark. Treat the figure as a vendor claim until independent evaluations reproduce it.

**Is DeepSeek V4 Pro open-weight?**

Yes — weights are distributed via DeepSeek's Hugging Face organization, which is what makes third-party hosting and self-hosting possible. Check the model card for license terms and hardware requirements before deploying.

## Sources

- [DeepSeek API Docs](https://api-docs.deepseek.com) — DeepSeek
- [DeepSeek (official site and privacy policy)](https://www.deepseek.com) — DeepSeek
- [deepseek-ai organization](https://huggingface.co/deepseek-ai) — Hugging Face
- [SWE-bench](https://www.swebench.com) — SWE-bench
- [OpenRouter](https://openrouter.ai) — OpenRouter
- [Together AI](https://www.together.ai) — Together AI
- [Fireworks AI](https://fireworks.ai) — Fireworks AI
- [Novita AI](https://novita.ai) — Novita AI
- [Hyperbolic](https://hyperbolic.xyz) — Hyperbolic
- [Cerebras](https://www.cerebras.ai) — Cerebras
- [SambaNova](https://sambanova.ai) — SambaNova
- [vLLM](https://github.com/vllm-project/vllm) — vLLM Project
- [SGLang](https://github.com/sgl-project/sglang) — SGLang Project
- [OpenAI Python SDK](https://github.com/openai/openai-python) — OpenAI


---

# How to Run GLM-5.3 and GLM-5.2 via API in 2026 (Hosts Compared)

*By DeAI Newsroom · 2026-08-24 · 9 min read*

Canonical: https://www.deai.org/news/run-glm-5-api

> **Summary:** GLM-5.3 and GLM-5.2 are MIT-licensed open-weight models you can call through Z.ai's official API, third-party hosts, or your own hardware. GLM-5.3 shipped on 2026-08-14, so third-party availability still lags the official endpoint.

GLM-5.3 and GLM-5.2 can be called through any OpenAI-compatible endpoint: point your client at Z.ai's official API or a third-party host, swap the base URL, and set the model slug. GLM-5.3 shipped on 2026-08-14 (as of 2026-08-20), so the official endpoint carries it first; third-party hosts follow on their own schedules.

## Key takeaways

- **2 models, 1 license:** GLM-5.3 and GLM-5.2 are both MIT-licensed; commercial API use, modification, and self-hosting are permitted.
- **GLM-5.3 released 2026-08-14** (as of 2026-08-20): expect a lag before third-party hosts list it; GLM-5.2 has broader third-party coverage today.
- **4 hosting routes:** Z.ai's official API, hosted third-party providers, decentralized marketplaces, or self-hosting. All four are callable from the same OpenAI-compatible client.
- **Switching hosts is a 3-line change:** base URL, API key, and model slug. No SDK rewrite.
- **0 verified per-token prices in this guide:** pricing is set per host and moves frequently, so this guide points to each host's pricing page instead of quoting numbers that go stale.

## GLM-5.3 vs GLM-5.2: what's the difference?

GLM-5.3 is the latest point release in Z.ai's GLM-5 line, released on 2026-08-14 (as of 2026-08-20). GLM-5.2 is its immediate predecessor and remains widely available. Both are part of the same MIT-licensed, open-weight family, so everything in this guide (endpoints, code, licensing) applies to both.

What a point release means in practice: the architecture and integration surface stay the same, while Z.ai's release notes describe whatever capability and behavior changes ship with the new checkpoint. This guide doesn't quote benchmark deltas. Check Z.ai's announcement and model card for the claims, then run your own prompts against both versions before committing. Point releases can shift behavior on specific tasks in ways headline scores don't capture.

The practical decision usually comes down to availability, not quality:

- **Starting a new project?** Default to GLM-5.3 on Z.ai's official endpoint, where it has been live since launch day.
- **Already committed to a third-party host?** Check whether it lists GLM-5.3 yet. If it only lists GLM-5.2, that is a stable, fully supported target, not a compromise you need to fix today.
- **In production?** Pin the exact model version in your config. Never point production traffic at a floating "latest" alias; point releases are exactly the kind of change that silently alters outputs.

The [model catalog](/models) tracks both checkpoints and where they are listed as hosts add them.

## Is the GLM-5 line really MIT-licensed?

Yes. Z.ai publishes the GLM-5 line (including GLM-5.3 and GLM-5.2) under the [MIT license](https://opensource.org/licenses/MIT), as stated on its model cards in the [zai-org Hugging Face organization](https://huggingface.co/zai-org). MIT is the most permissive option in common use: it allows commercial use, modification, redistribution, and self-hosting, with no copyleft obligations.

Two nuances worth understanding:

1. **The license covers the model, not the service.** MIT governs the weights and code you download. When you call a hosted API (Z.ai's or anyone else's), you are also bound by that provider's terms of service, acceptable-use policy, and privacy policy. Those are separate documents, and they differ per host.
2. **Verify per variant.** License fields are set per model card. Before you deploy a specific checkpoint, quantization, or fine-tune, open its card and confirm the license field says MIT. Community fine-tunes can carry different terms.

For API consumers, the license's biggest practical effect is competition: because anyone may serve the weights, you are not locked into the model creator's endpoint. That is what makes the rest of this article possible.

## Where can you run GLM-5 via API? Hosts compared

There are four routes, and they are not mutually exclusive. Many teams use the official endpoint for evaluation and a different host in production.

### Option 1: Z.ai's official API

[Z.ai](https://z.ai) operates the first-party API for the GLM line. It carries new releases first (GLM-5.3 has been available there since 2026-08-14) and exposes an OpenAI-compatible chat-completions interface, so standard SDKs work. You bill directly with Z.ai, and your prompts are handled under Z.ai's privacy and retention policies. As with any provider, those policies are the company's own statements; read the current versions rather than relying on summaries, including this one.

### Option 2: Third-party hosted providers and aggregators

Because the weights are MIT-licensed, independent inference providers can serve the GLM-5 line. Providers that have historically listed open-weight GLM models include Together AI, Fireworks, Novita, and DeepInfra. Aggregators like [OpenRouter's model catalog](https://openrouter.ai/models) let you see which backends currently list a given checkpoint and route requests to them through one API key.

The trade-offs: you consolidate billing with infrastructure you may already use, and you can sometimes pick a serving region. In exchange, brand-new releases arrive on the provider's schedule, not Z.ai's, so GLM-5.3 coverage varies day to day right now, while GLM-5.2 listings are more established. Logging and retention terms are set per provider and are policy statements, not independently verified facts; if data handling matters to your workload, compare the written policies before choosing.

### Option 3: Decentralized marketplaces

A third route is a decentralized inference marketplace, where independent operators serve open-weight models and routing and pricing emerge from the marketplace rather than a single company's price list. Morpheus is one example of a decentralized inference marketplace in this category. The evaluation criteria are identical to any other host: confirm which GLM checkpoints are currently listed, check the stated data-handling terms of the route you would actually use (again, stated policies rather than verified facts), and test latency from your region.

### Option 4: Self-hosting

MIT licensing plus public weights means you can skip hosted APIs entirely. Download the checkpoint from Hugging Face and serve it with an OpenAI-compatible server such as vLLM or SGLang, and your existing client code works unchanged against localhost. You get complete control over prompts and logs, and your cost becomes GPU time rather than per-token billing. The model card is the authoritative source for checkpoint size and serving requirements. Check it before provisioning hardware.

### How the four routes compare

| Route | New-release access | Billing | Data control | Ops burden |
|---|---|---|---|---|
| Z.ai official API | First (5.3 live since 2026-08-14) | Direct with Z.ai | Z.ai's stated policies | None |
| Third-party hosts | Provider's schedule; 5.2 broader today | Consolidated with existing infra | Per-provider stated policies | None |
| Decentralized marketplaces | Varies by listed operators | Marketplace-set | Per-route stated terms | Low |
| Self-host | Immediate (weights are public) | GPU costs | Full (your hardware) | Highest |

## How do you call the GLM-5 API? OpenAI-compatible quickstart

Every route above speaks the OpenAI chat-completions schema, so the integration is identical everywhere. The only host-specific values are the base URL, the API key, and the exact model slug.

**Step 1: get credentials.** Create an API key on your chosen host and find its OpenAI-compatible base URL (it usually ends in `/v1`).

**Step 2: set environment variables** so switching hosts later means changing config, not code:

```bash
export GLM_BASE_URL="https://your-host.example/v1"
export GLM_API_KEY="sk-..."
```

**Step 3: call it from Python** with the standard `openai` package:

```python
import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["GLM_API_KEY"],
    base_url=os.environ["GLM_BASE_URL"],
)

resp = client.chat.completions.create(
    model="glm-5.3",  # copy the exact slug from your host's model list
    messages=[
        {"role": "user", "content": "Explain the MIT license in one sentence."}
    ],
    temperature=0.2,
)
print(resp.choices[0].message.content)
```

**The same call in curl:**

```bash
curl "$GLM_BASE_URL/chat/completions" \
  -H "Authorization: Bearer $GLM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "glm-5.3",
    "messages": [
      {"role": "user", "content": "Explain the MIT license in one sentence."}
    ]
  }'
```

Three practitioner notes:

- **Model slugs differ per host.** One provider's `glm-5.3` is another's `zai-org/GLM-5.3`. Copy the identifier verbatim from the host's model list. A `404` or "model not found" error almost always means a wrong slug or that the host doesn't list that checkpoint yet.
- **Streaming works the same way.** Pass `stream=True` in the Python client or `"stream": true` in the JSON body, exactly as with any OpenAI-compatible server.
- **Keep the base URL in config, not code.** That one habit is what turns "switch hosts" from a migration into a deploy.

## What does GLM-5.2 cost per token?

Honest answer: there is no single number to quote. Pricing is set independently by each host and changes frequently, and DeAI News has no verified GLM-5.2 per-token prices as of 2026-08-20. Rather than publish figures that would be stale within weeks, here is how to price it yourself in five minutes:

- **Check each host's pricing page directly**: Z.ai for the official endpoint, and each third-party provider for theirs. Aggregators like OpenRouter display per-provider input and output rates side by side, which is the fastest comparison.
- **Expect this class of open-weight model to cost a fraction of frontier closed APIs.** That is a qualitative pattern, not a verified figure for any specific host.
- **Compare the full bill, not the headline rate.** Input and output tokens are usually priced asymmetrically, and caching, batch, or volume discounts can matter more than the list price for chatty workloads. Estimate your real input/output ratio before comparing.

## Choosing a host: a 60-second checklist

- Need **GLM-5.3 today**? Start on Z.ai's official endpoint.
- Already paying a **third-party inference provider**? Check its model list for the GLM-5 line before opening a new account.
- Have **data-handling requirements**? Read each candidate's current retention policy, and treat all of them as provider statements, not verified guarantees.
- Want **maximum control**? Self-host the MIT-licensed weights.
- Whoever you pick: **pin the model version**, keep the base URL in config, and run a handful of your own representative prompts before and after any switch.

For a wider view beyond the GLM family, see the guide to the [best open-source LLM APIs](/best-open-source-llm-api), which applies the same criteria across providers.

## FAQ

### What's the difference between GLM-5.3 and GLM-5.2?

GLM-5.3 is the newer point release in Z.ai's MIT-licensed GLM-5 line, released 2026-08-14. Both are open-weight; Z.ai's release notes detail capability changes. GLM-5.2 currently appears on more third-party hosts.

### Is GLM-5 released under the MIT license?

Yes. Z.ai publishes the GLM-5 line, including GLM-5.3 and GLM-5.2, under the MIT license per its model cards. MIT permits commercial use, modification, and self-hosting. Verify the license field on the exact Hugging Face variant you deploy.

### What are the alternatives to the official Z.ai API?

Third-party hosted providers and aggregators (OpenRouter, Together, Fireworks, Novita, DeepInfra), decentralized inference marketplaces such as Morpheus, or self-hosting the MIT-licensed weights. GLM-5.3 availability varies. Check each host's model list.

### What is the GLM-5.2 price per token?

Pricing is set per host and changes frequently; DeAI News has no verified GLM-5.2 token prices as of 2026-08-20. Models in this class typically cost a fraction of frontier closed APIs. Check each provider's pricing page for current rates.

### Do I need to rewrite my code to switch GLM-5 hosts?

Usually not. Most GLM-5 hosts expose an OpenAI-compatible endpoint, so switching means changing the base URL, API key, and model slug. Confirm the exact model identifier in the new host's documentation.

## FAQ

**GLM-5.3 vs GLM-5.2 — what's the difference?**

GLM-5.3 is the newer point release in Z.ai's MIT-licensed GLM-5 line, released 2026-08-14. Both are open-weight; Z.ai's release notes detail capability changes. GLM-5.2 currently appears on more third-party hosts.

**Is GLM-5 released under the MIT license?**

Yes. Z.ai publishes the GLM-5 line, including GLM-5.3 and GLM-5.2, under the MIT license per its model cards. MIT permits commercial use, modification, and self-hosting. Verify the license field on the exact Hugging Face variant you deploy.

**What are the alternatives to the official Z.ai API?**

Third-party hosted providers and aggregators (OpenRouter, Together, Fireworks, Novita, DeepInfra), decentralized inference marketplaces such as Morpheus, or self-hosting the MIT-licensed weights. GLM-5.3 availability varies — check each host's model list.

**What is the GLM-5.2 price per token?**

Pricing is set per host and changes frequently; DeAI News has no verified GLM-5.2 token prices as of 2026-08-20. Models in this class typically cost a fraction of frontier closed APIs. Check each provider's pricing page for current rates.

**Do I need to rewrite my code to switch GLM-5 hosts?**

Usually not. Most GLM-5 hosts expose an OpenAI-compatible endpoint, so switching means changing the base URL, API key, and model slug. Confirm the exact model identifier in the new host's documentation.

## Sources

- [Z.ai — GLM API platform](https://z.ai) — Z.ai
- [zai-org on Hugging Face (GLM model cards and weights)](https://huggingface.co/zai-org) — Hugging Face
- [OpenRouter model catalog](https://openrouter.ai/models) — OpenRouter
- [Together AI](https://www.together.ai) — Together AI
- [Fireworks AI](https://fireworks.ai) — Fireworks AI
- [Novita AI](https://novita.ai) — Novita AI
- [DeepInfra](https://deepinfra.com) — DeepInfra
- [The MIT License](https://opensource.org/licenses/MIT) — Open Source Initiative
- [OpenAI API reference (chat completions)](https://platform.openai.com/docs/api-reference) — OpenAI


---

# How to Run gpt-oss-120b via API in 2026 (Apache 2.0)

*By DeAI Newsroom · 2026-08-24 · 8 min read*

Canonical: https://www.deai.org/news/run-gpt-oss-120b

> **Summary:** gpt-oss-120b is OpenAI's 117B-parameter open-weight mixture-of-experts model, runnable behind any OpenAI-compatible API or self-hosted on a single 80 GB GPU.

You can run gpt-oss-120b behind any OpenAI-compatible API by changing two lines (the base URL and API key) or self-host the weights on a single 80 GB GPU. OpenAI's 117B-parameter mixture-of-experts model ships under Apache 2.0, so you pay no per-token license fees either way.

## Key takeaways

- Two integration lines, `base_url` and `api_key`, move any OpenAI SDK client to a gpt-oss-120b endpoint; the model id is usually `gpt-oss-120b` or `openai/gpt-oss-120b`.
- Self-hosting floor: OpenAI says the MXFP4-quantized weights (117B total, ~5.1B active per token) fit on one 80 GB GPU; community quantizations go lower at some quality cost.
- License: Apache 2.0. Commercial use, modification, and redistribution are permitted with zero usage restrictions in the license text; your provider's acceptable-use policy still applies on managed hosts.
- Reasoning effort (low / medium / high) is the main quality-vs-latency dial on gpt-oss models.
- Managed gpt-oss-120b APIs typically price at a fraction of frontier-API rates. Compare per-token prices on each provider's pricing page, current as of 2026-08-20.

## What is gpt-oss-120b?

gpt-oss-120b is the larger model in OpenAI's open-weights line, released alongside the smaller gpt-oss-20b. In its announcement, OpenAI described the pair as its first open-weight language models since GPT-2. The release put Apache 2.0-licensed weights on Hugging Face for anyone to download, inspect, fine-tune, and serve.

Per the model card, gpt-oss-120b is a mixture-of-experts transformer with 117 billion total parameters but only about 5.1 billion active per token. That sparsity is why a model with "120b" in the name is cheap to serve relative to dense models of similar total size: each token only touches a small slice of the network. The card also lists a long native context window (128k tokens), though many API providers cap context lower, so check the per-provider limit.

Two design details matter in practice. First, gpt-oss models were trained in OpenAI's "harmony" response format, which structures reasoning, tool calls, and final answers. Any serious serving stack applies this chat template for you, but raw-completions users must format it themselves. Second, the model exposes a reasoning-effort setting (low, medium, high) that trades latency and token spend against answer quality. DeAI's [model profiles](/models) track gpt-oss-120b alongside the rest of the open-weight field if you want a side-by-side spec view.

## What are gpt-oss-120b's hosting requirements?

For self-hosting, OpenAI says the default MXFP4 quantization fits on a single 80 GB GPU (H100 or A100 80 GB class). Plan for on the order of 60 GB of disk for the weights, plus VRAM headroom for the KV cache. Long-context, high-concurrency production deployments will want more than the bare floor, either via a second GPU with tensor parallelism or by capping context and batch size.

If one 80 GB card is out of reach, you have three realistic options:

- **Community quantizations.** Lower-bit builds shrink VRAM requirements further, at some quality cost. Evaluate on your own prompts before committing.
- **gpt-oss-20b.** The smaller sibling targets single consumer-grade GPUs and is a reasonable stand-in during development.
- **CPU offload.** Possible with llama.cpp-style stacks, but generally too slow for interactive workloads at the 120b size.

For serving software, vLLM is the common choice for throughput-oriented deployments; Ollama is the simplest single-node path. Both handle the harmony chat template and expose an OpenAI-compatible endpoint.

## Which providers host gpt-oss-120b?

If you don't want to own GPUs, managed inference is the faster path. As of 2026-08-20, gpt-oss-120b is broadly available across serverless providers such as Together, Fireworks, and Groq; aggregators like OpenRouter that route across multiple backends; privacy-focused providers such as Venice; and Morpheus, a decentralized inference marketplace. Lineups shift frequently, so treat this as a starting list, not a ranking.

Evaluate every option on identical criteria:

1. **Exact model id and quantization served.** Some hosts run the default MXFP4 build, others serve different quants.
2. **Context cap and throughput limits** actually enforced on your tier.
3. **Price per million tokens** on the provider's published pricing page.
4. **Region and latency** relative to your users.
5. **Data-retention policy.** Note that zero-retention and similar privacy promises are policy statements by the provider, not independently verified facts. Weigh them against your compliance requirements.
6. **Status page and historical uptime.**

DeAI's roundup of the [best open-source LLM APIs](/best-open-source-llm-api) applies this checklist across the major hosts.

## How do you call gpt-oss-120b from code?

Nearly every host exposes an OpenAI-compatible API, so integration is a base-URL swap against the official SDK:

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://api.your-provider.example/v1",  # your provider's endpoint
    api_key="YOUR_API_KEY",
)

response = client.chat.completions.create(
    model="gpt-oss-120b",  # some providers use "openai/gpt-oss-120b"
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain mixture-of-experts routing in one paragraph."},
    ],
    # reasoning_effort="medium",  # low | medium | high, if your provider supports it
)

print(response.choices[0].message.content)
```

The equivalent curl call:

```bash
curl https://api.your-provider.example/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-oss-120b",
    "messages": [
      {"role": "user", "content": "Explain mixture-of-experts routing in one paragraph."}
    ]
  }'
```

Practical notes: the model id string varies by provider, so copy it from their docs. `reasoning_effort` support also varies. Where it isn't exposed as a parameter, the model card describes setting the effort level in the system prompt instead. Streaming, retries, and error handling behave exactly as they do for any other OpenAI-compatible backend, which is the point: gpt-oss-120b can slot into existing OpenAI client code without a rewrite.

## How do you self-host gpt-oss-120b?

With vLLM on a suitable GPU node:

```bash
# Requires roughly 80 GB of VRAM for the default quantization
vllm serve openai/gpt-oss-120b --port 8000
```

With Ollama for a simpler single-machine setup:

```bash
ollama pull gpt-oss:120b
ollama serve  # exposes an OpenAI-compatible API at http://localhost:11434/v1
```

Then point the same Python client at your own endpoint (`base_url="http://localhost:8000/v1"` for vLLM) with any placeholder API key unless you've configured authentication. The client code doesn't change; only the URL does.

Two production caveats. First, if you bypass a chat-template-aware server and call raw completions, you must render the harmony format yourself. OpenAI's reference library is on GitHub, but most teams should simply use vLLM or Ollama and let them handle it. Second, never expose an unauthenticated inference port to the network: put a reverse proxy with TLS and auth in front, tune vLLM's batching and `max-num-seqs` for your traffic, and monitor GPU utilization and queue depth from day one.

## What does the gpt-oss usage policy actually say?

This is where confusion is common, so separate the three layers:

1. **The license.** gpt-oss weights are Apache 2.0. The license permits commercial use, modification, and redistribution, includes a patent grant, has no copyleft and, unlike some other "open" model licenses, no field-of-use restrictions or user-count clauses. It does not grant rights to OpenAI's trademarks.
2. **OpenAI's usage policies.** These are contractual terms that bind users of OpenAI's own hosted services. If you self-host the weights, your relationship with the model is governed by the Apache 2.0 license, not OpenAI's service terms. Many teams still treat OpenAI's published policies as a sensible baseline for their own acceptable-use rules.
3. **Your provider's terms.** If you use a managed host, whether centralized or a decentralized inference marketplace, that platform's acceptable-use policy applies to your traffic, and policies differ. Read them before you build.

Finally, license freedom is not a liability shield: the laws of your jurisdiction and your industry's regulations apply to whatever you build regardless of where the weights came from. None of this is legal advice.

## gpt-oss-120b vs GPT-5.5: which should you pick?

GPT-5.5 is OpenAI's closed, frontier-tier model; gpt-oss-120b is the open-weight line. Without inventing numbers, the honest framing is this: frontier proprietary models generally lead on the hardest reasoning, coding, and agentic tasks, and you should verify the current gap on public leaderboards and, better, on an eval set drawn from your own workload.

Choose **gpt-oss-120b** when you need downloadable weights (fine-tuning, quantization, auditing), data control (self-hosted inference means prompts never leave your infrastructure), predictable unit costs at high volume, or portability across providers. Choose **GPT-5.5** when peak capability matters more than control and you'd rather own zero operations. Many production teams run both: a frontier model for the hardest or most sensitive reasoning steps, gpt-oss-120b, typically at a fraction of frontier-API pricing, for high-volume routine traffic, with a router in between. The OpenAI-compatible interface shared by both makes that split a configuration detail rather than an engineering project.

## FAQ

### What is the gpt-oss usage policy?

gpt-oss weights ship under Apache 2.0, which itself imposes no usage restrictions. OpenAI's usage policies bind OpenAI-hosted services; third-party and decentralized hosts apply their own acceptable-use rules, so check your provider's terms.

### How does gpt-oss-120b compare to GPT-5.5?

GPT-5.5 is OpenAI's closed frontier model and is generally stronger on the hardest reasoning and coding tasks. gpt-oss-120b trades some peak capability for downloadable weights, fine-tuning freedom, data control, and typically a fraction of frontier-API pricing.

### What are gpt-oss hosting requirements?

OpenAI's model card describes a 117B-parameter MoE with roughly 5.1B active parameters per token; OpenAI says the MXFP4-quantized weights fit on one 80 GB GPU. Community quantizations and CPU offload trim hardware further at a speed or quality cost.

### Can you use gpt-oss-120b commercially?

Yes. Apache 2.0 permits commercial use, modification, and redistribution without copyleft or per-token license fees. The license does not grant OpenAI trademark rights, and your hosting provider's own terms still apply.

## FAQ

**What is the gpt-oss usage policy?**

gpt-oss weights ship under Apache 2.0, which itself imposes no usage restrictions. OpenAI's usage policies bind OpenAI-hosted services; third-party and decentralized hosts apply their own acceptable-use rules, so check your provider's terms.

**How does gpt-oss-120b compare to GPT-5.5?**

GPT-5.5 is OpenAI's closed frontier model and is generally stronger on the hardest reasoning and coding tasks. gpt-oss-120b trades some peak capability for downloadable weights, fine-tuning freedom, data control, and typically a fraction of frontier-API pricing.

**What are gpt-oss hosting requirements?**

OpenAI's model card describes a 117B-parameter MoE with roughly 5.1B active parameters per token; OpenAI says the MXFP4-quantized weights fit on one 80 GB GPU. Community quantizations and CPU offload trim hardware further at a speed or quality cost.

**Can you use gpt-oss-120b commercially?**

Yes. Apache 2.0 permits commercial use, modification, and redistribution without copyleft or per-token license fees. The license does not grant OpenAI trademark rights, and your hosting provider's own terms still apply.

## Sources

- [openai/gpt-oss-120b model card](https://huggingface.co/openai/gpt-oss-120b) — Hugging Face
- [Introducing gpt-oss](https://openai.com/index/introducing-gpt-oss/) — OpenAI
- [OpenAI usage policies](https://openai.com/policies/usage-policies/) — OpenAI
- [Apache License, Version 2.0](https://www.apache.org/licenses/LICENSE-2.0) — Apache Software Foundation
- [openai/harmony](https://github.com/openai/harmony) — GitHub
- [vLLM documentation](https://docs.vllm.ai) — vLLM Project
- [Ollama](https://ollama.com) — Ollama
- [OpenRouter](https://openrouter.ai) — OpenRouter
- [Together AI](https://www.together.ai) — Together AI
- [Fireworks AI](https://fireworks.ai) — Fireworks AI
- [Groq](https://groq.com) — Groq


---

# How to Run Hermes 4 and Uncensored Fine-Tunes via API in 2026

*By DeAI Newsroom · 2026-08-24 · 8 min read*

Canonical: https://www.deai.org/news/run-hermes-4-uncensored-models

> **Summary:** Hermes 4 and uncensored fine-tunes like Dolphin are open weights, so any OpenAI-compatible host — or your own vLLM server — can serve them behind the same two-line client change.

You can run Nous Hermes 4 and uncensored fine-tunes like Dolphin through any OpenAI-compatible API by changing two lines of client code: the base URL and the API key. Pick a hosted provider that lists the model, or self-host the open weights with vLLM. This guide walks both paths.

## Key takeaways

- Hermes 4 ships in 2 open-weight sizes (70B and 405B), so the same checkpoints run on third-party hosted APIs or on your own GPUs.
- Switching providers is typically a 2-line change (`base_url` + `api_key`), because nearly every open-weight host speaks the OpenAI chat-completions schema.
- A 70B model needs roughly 2 bytes of VRAM per parameter at 16-bit precision (~140 GB before the KV cache); 4-bit quantization cuts that to about a quarter.
- "Uncensored" is a spectrum, not a binary: low-refusal claims on model cards are self-reported, and DeAI's refusal-index methodology scores actual compliance versus refusal across prompt categories rather than trusting labels.
- Hosted uncensored endpoints typically cost a fraction of frontier-API pricing, but retention and moderation terms differ per provider. Read the policy page, not the landing page.

## What exactly are Hermes 4 and "uncensored" fine-tunes?

Three families of models dominate the low-refusal niche, and they get there in different ways. Knowing which mechanism you're dealing with matters, because it determines how the model behaves under a system prompt and how much you should trust the label.

### Nous Hermes 4 70B and 405B

Hermes 4 is the fourth-generation open-weight instruct series from [Nous Research](https://nousresearch.com). According to the model cards, [Hermes 4 70B](https://huggingface.co/NousResearch/Hermes-4-70B) is post-trained from Meta's Llama 3.3 70B, while [Hermes 4 405B](https://huggingface.co/NousResearch/Hermes-4-405B) builds on Llama 3.1 405B. The series' headline feature is a hybrid reasoning mode: the model can produce extended chain-of-thought traces when steered, or answer directly when you want lower latency. Nous positions Hermes as a steerable, maximally helpful series. Treat that as the developer's positioning and verify it against your own prompt set, as you should with any vendor claim.

Because the weights are public, Hermes 4 shows up on many hosted inference catalogs, and you can also serve it yourself. That dual availability is the whole point of this guide.

### Dolphin fine-tunes

Dolphin is a long-running family of fine-tunes from Eric Hartford's [cognitivecomputations](https://huggingface.co/cognitivecomputations) project, built on various bases (Llama and Qwen variants, among others). The creator explicitly describes Dolphin as uncensored, with alignment moved into the system prompt so the deployer, not the lab, decides acceptable behavior. That description is documented on the model cards; it is still a self-reported claim, but it is at least a specific, checkable one: you can probe the system-prompt steerability yourself in an afternoon.

### Abliterated checkpoints

A third path is abliteration: a weight-level edit that removes the "refusal direction" identified by Arditi et al. in [*Refusal in Language Models Is Mediated by a Single Direction*](https://arxiv.org/abs/2406.11717), without retraining. Community publishers apply it to many popular bases. Abliterated models behave differently from fine-tuned ones; the edit can occasionally blunt instruction-following edge cases, so it is worth understanding the technique before committing to it in production. Our explainer on [abliterated and uncensored models](/abliterated-uncensored-models-explained) covers the mechanics and the trade-offs.

## Where can you get uncensored model API hosting?

Catalogs change weekly, so treat every provider's model page as the source of truth. Historically, Hermes and Dolphin family models have been listed by aggregators and serverless hosts such as [OpenRouter](https://openrouter.ai/models), [Featherless](https://featherless.ai) (which specializes in long-tail open weights), DeepInfra, Hyperbolic, Novita, and Together. Check each catalog for the exact variant and quantization currently served.

Evaluate candidates on identical criteria:

- **Exact model ID and variant.** "Hermes 4 70B" might mean full-precision, AWQ, or FP8 depending on the host. The served quantization affects output quality.
- **Context length actually exposed.** Hosts frequently cap context below the model's trained window to control costs.
- **Host-level moderation.** Some providers layer their own content filters on top of the weights. An uncensored model behind a filtered endpoint still refuses at the API layer. This is a property of the host, not the checkpoint.
- **Retention and privacy policy.** Read the data-policy page. Statements like "we don't store prompts" are policy commitments, not independently verified facts, whoever makes them.
- **Pricing page.** Hosted open-weight endpoints typically run a fraction of frontier-API pricing, but the spread between providers is wide enough to be worth an hour of comparison.

For a tracked, criteria-based comparison, see our roundup of the [best uncensored AI APIs](/best-uncensored-ai-api).

### Decentralized and privacy-focused options

If you prefer not to depend on a single company's infrastructure, decentralized networks route requests to independent operators. Chutes is one such network; Morpheus is a decentralized inference marketplace in the same category. Apply the same checklist as above (model ID, context, moderation, retention), since operator behavior can vary inside a network. Venice, a centralized provider, markets a no-retention privacy posture; as with any provider, that is a policy statement rather than a verified property. None of these structural differences exempts you from testing the endpoint yourself.

## How do you call Hermes 4 from an OpenAI-compatible API?

The integration pattern is identical across nearly every open-weight host: create an account, generate an API key, copy the exact model ID from the provider's catalog, and point your client at the provider's base URL. Model ID strings differ per host (one might use `NousResearch/Hermes-4-70B`, another `nous/hermes-4-70b`), so always copy rather than guess.

Python, using the standard `openai` client:

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://api.your-provider.example/v1",  # provider's OpenAI-compatible endpoint
    api_key="YOUR_PROVIDER_KEY",
)

resp = client.chat.completions.create(
    model="nous-hermes-4-70b",  # exact model ID varies by provider — copy from the catalog
    messages=[
        {"role": "system", "content": "You are a precise technical assistant."},
        {"role": "user", "content": "Explain mixture-of-experts routing in one paragraph."},
    ],
    temperature=0.7,
)
print(resp.choices[0].message.content)
```

The same call with curl:

```bash
curl https://api.your-provider.example/v1/chat/completions \
  -H "Authorization: Bearer YOUR_PROVIDER_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "nous-hermes-4-70b",
    "messages": [{"role": "user", "content": "Hello"}],
    "stream": false
  }'
```

Streaming, tool calling, and reasoning-mode toggles work through the same schema where the host supports them. Check the Hermes 4 model card for the creators' recommended sampling settings before you tune `temperature` by feel.

## How do you self-host Hermes 4 70B with vLLM?

Self-hosting trades a per-token bill for full custody: prompts never leave your hardware, and no host-side moderation layer exists. The arithmetic is straightforward. At 16-bit precision, plan on roughly 2 bytes of VRAM per parameter, about 140 GB for the 70B before the KV cache, which fits a pair of 80 GB data-center GPUs. A 4-bit quantized build (AWQ or GPTQ) drops weights to roughly 35–40 GB, within reach of a single 48 GB card or a pair of consumer 24 GB cards. The 405B is a different league: think a full 8-GPU node at 16-bit, or several GPUs at 4-bit. Hourly GPU marketplaces such as RunPod, Vast.ai, and Lambda rent all of these configurations; compare their current rate cards.

[vLLM](https://docs.vllm.ai) is the default serving stack because its continuous batching keeps throughput high under concurrent load, and it exposes an OpenAI-compatible server out of the box:

```bash
pip install vllm

vllm serve NousResearch/Hermes-4-70B \
  --max-model-len 8192 \
  --gpu-memory-utilization 0.92
```

For a quantized deployment, point `--model` at an AWQ checkpoint instead. Once the server is up, the client code from the previous section works unchanged. Just swap the base URL and use any placeholder key:

```python
client = OpenAI(base_url="http://localhost:8000/v1", api_key="not-needed")
```

That is the entire migration story: hosted and self-hosted endpoints are interchangeable behind the same interface, which is also your vendor lock-in insurance. For production, put the server behind authentication and a load balancer, and pin your vLLM version so upgrades are deliberate.

## How "uncensored" are these models in practice?

Less than the label implies, and the variance comes from three places. First, the mechanism: fine-tuned compliance (Dolphin), steerability-focused post-training (Hermes), and weight edits (abliteration) produce different refusal profiles, and none reduces refusals to literally zero. Second, the host: a provider's own moderation layer can reintroduce refusals regardless of the checkpoint. Third, the prompt itself: refusal rates are a distribution across categories, not a single number.

This is why DeAI's refusal-index methodology scores model responses across prompt categories instead of accepting "uncensored" as a label, and why creator claims, however well-documented on a model card, remain self-reported until probed. The practical move is a small eval of your own: 20–30 prompts spanning the categories you actually care about, run against each candidate endpoint, scored for compliance versus refusal. An afternoon of that beats any landing page.

## FAQ

### What is Nous Hermes 4 70B?

The mid-size model in Nous Research's Hermes 4 open-weight series, post-trained from Meta's Llama 3.3 70B per its model card. It exposes a hybrid reasoning mode and can be called from hosted OpenAI-compatible APIs or self-hosted with vLLM.

### Where can I find uncensored model API hosting?

Aggregators and serverless hosts such as OpenRouter, Featherless, DeepInfra, Hyperbolic, and Novita have listed Hermes and Dolphin family models; decentralized options include Chutes and Morpheus, a decentralized inference marketplace. Catalogs change often, so check each provider's model page.

### Can I run a Dolphin fine-tune via API?

Yes. Dolphin checkpoints are open weights, so any host that lists them can serve them behind an OpenAI-compatible endpoint, and you can deploy any Dolphin variant yourself with vLLM. The exact model ID differs per provider, so copy it from the provider's catalog.

### Are uncensored models truly refusal-free?

No. "Uncensored" describes a spectrum of lower-refusal tuning, and creators' claims are self-reported. Hosts may also layer on their own moderation. DeAI's refusal-index methodology scores responses across prompt categories; the practical test is your own prompt set.

## FAQ

**What is Nous Hermes 4 70B?**

The mid-size model in Nous Research's Hermes 4 open-weight series, post-trained from Meta's Llama 3.3 70B per its model card. It exposes a hybrid reasoning mode and can be called from hosted OpenAI-compatible APIs or self-hosted with vLLM.

**Where can I find uncensored model API hosting?**

Aggregators and serverless hosts such as OpenRouter, Featherless, DeepInfra, Hyperbolic, and Novita have listed Hermes and Dolphin family models; decentralized options include Chutes and Morpheus, a decentralized inference marketplace. Catalogs change often — check each provider's model page.

**Can I run a Dolphin fine-tune via API?**

Yes. Dolphin checkpoints are open weights, so any host that lists them can serve them behind an OpenAI-compatible endpoint, and you can deploy any Dolphin variant yourself with vLLM. The exact model ID differs per provider — copy it from the provider's catalog.

**Are uncensored models truly refusal-free?**

No. 'Uncensored' describes a spectrum of lower-refusal tuning, and creators' claims are self-reported. Hosts may also layer on their own moderation. DeAI's refusal-index methodology scores responses across prompt categories; the practical test is your own prompt set.

## Sources

- [NousResearch/Hermes-4-70B model card](https://huggingface.co/NousResearch/Hermes-4-70B) — Hugging Face
- [NousResearch/Hermes-4-405B model card](https://huggingface.co/NousResearch/Hermes-4-405B) — Hugging Face
- [Nous Research](https://nousresearch.com) — Nous Research
- [cognitivecomputations (Dolphin) on Hugging Face](https://huggingface.co/cognitivecomputations) — Hugging Face
- [vLLM documentation](https://docs.vllm.ai) — vLLM Project
- [OpenRouter model catalog](https://openrouter.ai/models) — OpenRouter
- [Featherless serverless inference](https://featherless.ai) — Featherless
- [Refusal in Language Models Is Mediated by a Single Direction](https://arxiv.org/abs/2406.11717) — arXiv


---

# How to Run Kimi K2.5 via API in 2026 (Hosts Compared)

*By DeAI Newsroom · 2026-08-24 · 8 min read*

Canonical: https://www.deai.org/news/run-kimi-k2-5-api

> **Summary:** Kimi K2.5 is served by Moonshot AI's first-party API and a half-dozen third-party hosts, nearly all behind OpenAI-compatible endpoints — reference pricing is about $0.60 per million input tokens and $3.00 per million output tokens as of 2026-08-20.

You can run Kimi K2.5 through any OpenAI-compatible API by changing two lines: the base URL and the API key. Moonshot AI's reference pricing is about $0.60 per million input tokens and $3.00 per million output tokens (as of 2026-08-20). This guide compares every host serving K2.5 and shows the exact code.

## Key takeaways

- Kimi K2.5's reference API pricing is roughly **$0.60 per million input tokens** and **$3.00 per million output tokens** (as of 2026-08-20), or about **$12** for a workload of 10M input plus 2M output tokens.
- K2.5 is served by first-party and third-party hosts: inference clouds, an aggregator, and a decentralized inference marketplace, nearly all behind OpenAI-compatible endpoints.
- Switching hosts is a two-line change (`base_url` + `api_key`); the model ID is the only other thing that sometimes differs.
- Hosted K2.5 typically costs a fraction of frontier-API rates such as GPT-5.2's; verify current numbers on each provider's pricing page before committing.
- List prices move frequently. Each provider's pricing page is the source of truth, and DeAI's [cheapest LLM API](/cheapest-llm-api) roundup tracks them in one place.

## What is Kimi K2.5?

Kimi K2.5 is Moonshot AI's open-weight entry in the Kimi K2 line. Because the weights are published through [Moonshot AI's Hugging Face organization](https://huggingface.co/moonshotai), you are not locked to one vendor: Moonshot runs a first-party API, and multiple independent hosts serve the same weights over their own infrastructure. Moonshot positions the K2 family at chat, coding, and agentic tool-use workloads; for architecture details and exact license terms, the model card is the authoritative source.

That open distribution is what makes host-shopping worthwhile. It's also why "the Kimi K2.5 API" is really several APIs with different prices, rate limits, and data terms. If you're evaluating the newer generation, our guide to [running Kimi K3 via API](/run-kimi-k3-api) covers the same ground for K3.

## How much does the Kimi K2.5 API cost?

Moonshot AI's reference pricing for K2.5 is roughly **$0.60 per million input tokens** and **$3.00 per million output tokens** (as of 2026-08-20). A worked example: a workload of 10M input tokens and 2M output tokens costs about (10 × $0.60) + (2 × $3.00) = **$12** at reference rates.

Third-party hosts set their own rates, and the billing models differ:

- **Serverless hosts** bill per token, usually with separate input and output rates.
- **Dedicated deployments** bill per GPU-hour, which can make sense at sustained volume.
- **Aggregators** pass through upstream pricing, sometimes with a fee on top.
- **Marketplaces** let rates float with provider supply.

We only republish figures we can verify against a canonical source. As of 2026-08-20, the reference rate above is the confirmed number, and third-party list prices for K2.5 move often enough that the pricing pages linked below are the reliable place to check. Also look past the headline per-token figure: prompt caching, batch discounts, and context-tier pricing can change your effective rate materially.

## Which hosts serve Kimi K2.5?

The table below lists the main places to get a K2.5 endpoint as of 2026-08-20. Availability changes, so confirm on each provider's model list before you build.

| Host | Category | Billing model | K2.5 pricing (as of 2026-08-20) |
|---|---|---|---|
| [Moonshot AI](https://platform.moonshot.ai/) (official) | First-party API | Per token | ~$0.60/M input, ~$3.00/M output (reference) |
| [Fireworks AI](https://fireworks.ai/pricing) | Inference cloud | Per token; dedicated options | See pricing page |
| Morpheus | Decentralized inference marketplace | Per token; rates set by independent providers | Varies by provider; check current marketplace listings |
| [Nebius AI Studio](https://nebius.com) | Inference cloud | Per token | See pricing page |
| [Novita AI](https://novita.ai) | Inference cloud | Per token | See pricing page |
| [OpenRouter](https://openrouter.ai/models) | Aggregator | Per token; pass-through from upstreams | Varies by upstream; see the K2.5 model page |
| [Together AI](https://www.together.ai/pricing) | Inference cloud | Per token; dedicated options | See pricing page |

A few notes on reading the table:

- **"Reference" pricing is Moonshot's first-party rate.** Third parties price independently of it, in either direction.
- **OpenRouter** is one API key over multiple upstream providers; its K2.5 model page shows which upstreams are live and what each charges per token.
- **Morpheus** is structured differently from the rest: it is a decentralized inference marketplace where independent providers bid to serve requests, so per-token rates and data-handling terms are set per provider rather than by a single company.
- Throughput and uptime figures on provider sites are self-reported claims; treat them as such until you can measure latency on your own prompts.

## How do you call the Kimi K2.5 API?

Almost every K2.5 host exposes an OpenAI-compatible chat-completions endpoint. If your code already talks to an OpenAI-style API, you're changing configuration, not code.

### 1. Get an API key

Sign up with your chosen host, create a key in its dashboard, and note two things: the **base URL** and the **exact model ID** for K2.5. Model IDs are not standardized across hosts: one may use a short name, another a namespaced ID. Copy it from the host's docs rather than guessing.

### 2. Call it with the OpenAI Python SDK

The [OpenAI Python SDK](https://github.com/openai/openai-python) works against any compatible endpoint:

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://YOUR-HOST-ENDPOINT/v1",  # the host's OpenAI-compatible URL
    api_key="YOUR_API_KEY",                     # from the host's dashboard
)

response = client.chat.completions.create(
    model="kimi-k2.5",  # exact model ID varies by host — check its docs
    messages=[
        {"role": "system", "content": "You are a concise assistant."},
        {"role": "user", "content": "Explain mixture-of-experts routing in one paragraph."},
    ],
    temperature=0.6,
    max_tokens=512,
)

print(response.choices[0].message.content)
```

### 3. Or use curl

```bash
curl -X POST "https://YOUR-HOST-ENDPOINT/v1/chat/completions" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "kimi-k2.5",
    "messages": [
      {"role": "system", "content": "You are a concise assistant."},
      {"role": "user", "content": "Explain mixture-of-experts routing in one paragraph."}
    ],
    "temperature": 0.6,
    "max_tokens": 512
  }'
```

### 4. Switch hosts by changing two lines

Because the API shape is identical across hosts, a multi-host setup is just configuration:

```python
import os
from openai import OpenAI

HOSTS = {
    "host_a": {"base_url": "https://host-a.example/v1", "model": "kimi-k2.5"},
    "host_b": {"base_url": "https://host-b.example/v1", "model": "moonshotai/kimi-k2.5"},
}

cfg = HOSTS["host_a"]  # swap to "host_b" to fail over
client = OpenAI(base_url=cfg["base_url"], api_key=os.environ["HOST_API_KEY"])
```

This is the practical answer to price movement: keep a second host's key in your environment and you can fail over, or A/B cost and latency, without touching application code.

## Kimi K2.5 vs GPT-5.2: what's the cost difference?

GPT-5.2 is OpenAI's frontier line, sold through OpenAI's first-party API; OpenAI publishes current per-token rates on its [pricing page](https://platform.openai.com/docs/pricing). K2.5 is open-weight and sold by many competing hosts. As a rule, hosted open-weight models land at a fraction of frontier-API per-token pricing, and K2.5's reference rate ($0.60/M input, $3.00/M output as of 2026-08-20) is consistent with that pattern.

The honest way to compare: take a representative week of your traffic (input tokens, output tokens, cache hit rate) and price it on both. Output tokens dominate most chat and agent workloads, so the output rate matters more than the input rate. Then weigh the non-price differences: K2.5 gives you multiple vendors and a self-host fallback; GPT-5.2 gives you a single first-party endpoint. On quality, don't treat anyone's benchmark table as final, including a provider's own. Run both models on a few hundred of your real prompts and score the outputs yourself.

## What should you check before picking a K2.5 host?

- **Exact model served.** Confirm the endpoint serves K2.5, not the older K2, and note the precise model ID.
- **Context and output limits actually enabled.** These vary between providers serving the same weights.
- **Data retention.** Zero-retention and "we don't train on your data" promises are policy statements, not independently verified facts. Read each provider's terms; on aggregators and marketplaces, establish which underlying operator actually processes your request.
- **Rate limits and throughput.** Provider-published tokens-per-second figures are self-reported claims.
- **Feature parity.** Tool calling, structured outputs/JSON mode, and streaming behave differently across hosts, so test the specific features your app uses.
- **Billing model.** Per-token serverless versus per-hour dedicated; sustained high volume can flip which is cheaper.
- **Exit cost.** Prefer hosts with OpenAI-compatible endpoints (everything in the table above qualifies) so switching stays a two-line change.

## FAQ

### How much does the Kimi K2.5 API cost?

Moonshot AI's reference pricing is roughly $0.60 per million input tokens and $3.00 per million output tokens (as of 2026-08-20). Third-party hosts set their own rates, so confirm each provider's pricing page before committing.

### Is Kimi K2.5 cheaper than GPT-5.2?

At its reference rates ($0.60/M input, $3.00/M output as of 2026-08-20), hosted K2.5 typically costs a fraction of frontier-API pricing. Check OpenAI's pricing page for current GPT-5.2 rates, and compare both on your real token mix.

### Is the Kimi K2.5 API OpenAI-compatible?

Yes. Moonshot's API and most third-party hosts expose OpenAI-compatible chat-completions endpoints, so switching is usually a base-URL and API-key change. Exact model IDs vary by host, so copy the ID from your provider's docs.

### Can I self-host Kimi K2.5 instead of using an API?

Yes. K2.5 is an open-weight release with weights distributed via Moonshot AI's Hugging Face organization. Self-hosting demands serious multi-GPU capacity; the model card lists hardware requirements and license terms.

## FAQ

**How much does the Kimi K2.5 API cost?**

Moonshot AI's reference pricing is roughly $0.60 per million input tokens and $3.00 per million output tokens (as of 2026-08-20). Third-party hosts set their own rates, so confirm each provider's pricing page before committing.

**Is Kimi K2.5 cheaper than GPT-5.2?**

At its reference rates ($0.60/M input, $3.00/M output as of 2026-08-20), hosted K2.5 typically costs a fraction of frontier-API pricing. Check OpenAI's pricing page for current GPT-5.2 rates, and compare both on your real token mix.

**Is the Kimi K2.5 API OpenAI-compatible?**

Yes. Moonshot's API and most third-party hosts expose OpenAI-compatible chat-completions endpoints, so switching is usually a base-URL and API-key change. Exact model IDs vary by host — copy the ID from your provider's docs.

**Can I self-host Kimi K2.5 instead of using an API?**

Yes. K2.5 is an open-weight release with weights distributed via Moonshot AI's Hugging Face organization. Self-hosting demands serious multi-GPU capacity; the model card lists hardware requirements and license terms.

## Sources

- [Moonshot AI Platform](https://platform.moonshot.ai/) — Moonshot AI
- [Moonshot AI on Hugging Face](https://huggingface.co/moonshotai) — Hugging Face
- [Fireworks AI Pricing](https://fireworks.ai/pricing) — Fireworks AI
- [Nebius](https://nebius.com) — Nebius
- [Novita AI](https://novita.ai) — Novita AI
- [OpenRouter Models](https://openrouter.ai/models) — OpenRouter
- [Together AI Pricing](https://www.together.ai/pricing) — Together AI
- [OpenAI API Pricing](https://platform.openai.com/docs/pricing) — OpenAI
- [OpenAI Python SDK](https://github.com/openai/openai-python) — OpenAI


---

# How to Run Kimi K3 via API in 2026 (Price, Context, License Caveats)

*By DeAI Newsroom · 2026-08-24 · 8 min read*

Canonical: https://www.deai.org/news/run-kimi-k3-api

> **Summary:** Kimi K3 is Moonshot AI's ~2.8-trillion-parameter open-weight MoE (~104B active, as reported by Moonshot as of 2026-08-20), callable from any OpenAI-compatible endpoint.

You can run Kimi K3 through any OpenAI-compatible API by changing the base URL and model ID. No new SDK required. K3 is a ~2.8-trillion-parameter mixture-of-experts model with ~104B active parameters, as reported by Moonshot AI as of 2026-08-20. This guide covers pricing, context, license caveats, and every realistic way to run it.

## Key takeaways

- Kimi K3 is ~2.8T total parameters with ~104B active per token (as reported by Moonshot AI, as of 2026-08-20). It's a sparse MoE, so per-token compute is a fraction of its total size, but you still need to fit all weights in memory to self-host.
- Any OpenAI-compatible client works: swap `base_url` and the model ID, keep your existing code.
- Self-hosting requires roughly 2.8 TB of GPU memory at FP8 (about double at BF16) before KV cache: multi-node territory.
- There is no single K3 price; each host sets its own rates. Open-weight MoE APIs typically run at a fraction of frontier closed-API pricing, but compare published pricing pages. DeAI News had no verified K3 price as of 2026-08-20.
- License: read the model card before commercial use. Recent K-series releases shipped under a modified MIT license with an attribution clause at very large scale.

## What is Kimi K3?

Kimi K3 is Moonshot AI's flagship open-weight model and the successor in the K series that includes K2 and K2.6. It is a sparse mixture-of-experts (MoE) model: ~2.8 trillion total parameters, with ~104 billion activated per token, as reported by Moonshot AI as of 2026-08-20.

That architecture matters for two practical reasons. First, inference cost per token scales with the *active* parameters, not the total, which is why hosted K3 endpoints can be priced far below what a dense 2.8T model would cost to serve. Second, the *total* parameter count still determines the memory footprint, which is why self-hosting is a datacenter-scale project (more on that below).

Moonshot positions K3 at coding, reasoning, and agentic tool-use workloads, per its launch materials. Treat that as a vendor positioning statement and validate against your own tasks. Weights are published on Hugging Face under the moonshotai organization, and you can track availability and metadata on DeAI's [model pages](/models).

## Is Kimi K3's license OK for commercial use?

This is the caveat most teams skip and shouldn't. "Open weight" does not automatically mean "open source". Our explainer on [open-weight vs. open-source licenses](/open-weight-vs-open-source) covers the distinction and why it matters for compliance.

Recent releases in the Kimi K series have used a modified MIT license: broadly permissive, commercial use allowed, but with an additional clause requiring prominent attribution for products above very large usage or revenue thresholds. Whether K3 carries identical terms is something you must verify on the K3 model card itself, because Moonshot can change terms per release.

A practical pre-deployment checklist:

- **Read the LICENSE file in the official repo** itself, not a summary or a blog post.
- **If you redistribute weights or fine-tunes**, check the naming and attribution requirements for derivatives.
- **If you operate at consumer-product scale**, have counsel review any threshold-based clauses before launch, not after.
- **If you use a hosted API**, the provider's terms of service apply on top of the model license.

For most ordinary commercial workloads (internal tools, SaaS features, agents), permissive open-weight licenses of this type are workable. The risk is in assuming, not in reading.

## How does Kimi K3 benchmark against its peers?

Only cited numbers belong here, and as of 2026-08-20 the verified public anchor is the architecture itself: ~2.8T total / ~104B active parameters, as reported by Moonshot AI.

Moonshot's launch materials report results placing K3 at or near the top of open-weight models on coding and agentic suites. Those are vendor-run evaluations, so weight them accordingly. For independent signal, community-driven rankings such as LMArena are the standard first stop, and the serious move is to re-run your own eval harness against K3 before migrating production traffic.

One dimension most benchmark tables ignore is refusal behavior: how often a model declines benign requests. DeAI's refusal-index methodology scores exactly that, and results are published on each model's page as they complete. Worth checking for K3 if you run user-facing agents where over-refusal is a real cost.

## Where can you run Kimi K3?

### First-party and hosted APIs

Catalogs change weekly, so treat this as a checklist of places to look rather than a permanent directory:

- **Moonshot AI's first-party platform** (platform.moonshot.ai) is the reference endpoint, typically first to receive updates and full feature support such as tool calling.
- **Aggregators** such as OpenRouter route K3-class models across multiple underlying hosts through one API, useful for price comparison and failover.
- **Hosted inference providers** including Together AI, Fireworks AI, Groq, DeepInfra, Novita, Nebius, Baseten, and SiliconFlow have all carried recent open-weight flagship releases; confirm K3 in each provider's model catalog.
- **Decentralized options** include Morpheus, a decentralized inference marketplace, where independent operators serve open-weight models; the same evaluation criteria apply as to any host.

Evaluate every option on identical criteria: does the catalog actually list K3, what are the published per-token rates, what context length does the endpoint expose (hosts sometimes cap it below the model maximum), which regions are served, and what is the data-retention policy. Note that zero-retention and "operators can't see prompts" statements are policy claims, not independently verified facts. If privacy is a hard requirement, prefer providers with published policies you find credible, or self-host.

### What are Kimi K3's self-host requirements?

Here the verified anchor does the math for you. At ~2.8T parameters:

- **FP8 (1 byte/param):** ~2.8 TB of memory for weights alone.
- **BF16 (2 bytes/param):** ~5.6 TB for weights alone.
- **Plus KV cache**, which grows with context length and concurrent requests. That's significant for a long-context model.

In hardware terms, that is multiple 8-GPU nodes of 80 GB-class accelerators at FP8, connected by fast interconnect (NVLink within nodes, InfiniBand across them), using tensor and expert parallelism. vLLM and SGLang are the standard serving stacks for large MoE models, and both document multi-node deployment. Community quantization builds can shrink the footprint further, with the usual quality trade-offs, so validate on your workload.

Honest assessment: self-hosting K3 is a six-figure infrastructure commitment. Unless you have strict data-residency requirements or very large steady-state volume, a hosted API is the rational starting point.

## How do you call the Kimi K3 API?

Every host above exposes an OpenAI-compatible endpoint, so integration is a two-line change. Model IDs vary by provider (e.g., `kimi-k3`, `moonshotai/Kimi-K3-...`), so copy the exact string from your provider's docs.

**Python:**

```python
from openai import OpenAI

client = OpenAI(
    api_key="YOUR_PROVIDER_KEY",
    base_url="https://YOUR-PROVIDER-ENDPOINT/v1",  # the only line that changes
)

resp = client.chat.completions.create(
    model="kimi-k3",  # exact model ID from your provider's catalog
    messages=[{"role": "user", "content": "Explain MoE routing in one paragraph."}],
    stream=True,
)
for chunk in resp:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")
```

**curl:**

```bash
curl https://YOUR-PROVIDER-ENDPOINT/v1/chat/completions \
  -H "Authorization: Bearer YOUR_PROVIDER_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "kimi-k3",
    "messages": [{"role": "user", "content": "Explain MoE routing in one paragraph."}],
    "stream": true
  }'
```

Provider quirks to check before production: supported parameter ranges, tool-calling schema support, whether `max_tokens` defaults are sane for a long-context model, and rate limits on the tier you're on.

## Kimi K3 vs K2.6: should you switch?

K3 is the next generation at a larger scale (~2.8T total / ~104B active parameters as reported by Moonshot AI as of 2026-08-20), while K2.6 remains available on many endpoints. Switching is mechanically trivial (a model-ID change on the same base URL), but do three things first: diff pricing between the two on your host, confirm the context window you need is exposed on the K3 endpoint, and re-run your eval suite, because generation-to-generation behavior shifts, including refusal patterns, can break prompts that were tuned against K2.6. If K2.6 meets your quality bar at a lower price on your provider, there is no forced march.

## FAQ

**Is Kimi K3's license OK for commercial use?**
Read the LICENSE file on the official Hugging Face model card before deploying. Recent Kimi K-series releases used a modified MIT license that permits commercial use but adds an attribution requirement for very large products. Terms can change per release.

**How much does the Kimi K3 API cost?**
There is no single price: each host sets its own per-token rates, and DeAI News had no verified K3 price as of 2026-08-20. Open-weight MoE APIs typically cost a fraction of frontier closed APIs. Compare providers' published pricing pages.

**What's the difference between Kimi K3 and K2.6?**
K3 is Moonshot's next generation at ~2.8T total and ~104B active parameters, as reported by Moonshot AI as of 2026-08-20. Migration is usually a model-ID swap on the same endpoint, but re-run your own evals, because behavior and refusals can shift.

**What do you need to self-host Kimi K3?**
Roughly 2.8 TB of GPU memory at FP8 (about double at BF16) for weights alone, plus KV cache. That's multi-node cluster territory with fast interconnect. vLLM and SGLang are the usual serving stacks. Most teams use a hosted API instead.

**What is Kimi K3's context length?**
Confirm the exact window on the model card, and on your provider's endpoint, since hosts sometimes cap context below the model maximum. Long prompts raise both latency and cost, so size your context budget before committing.

## FAQ

**Is Kimi K3's license OK for commercial use?**

Read the LICENSE file on the official Hugging Face model card before deploying. Recent Kimi K-series releases used a modified MIT license that permits commercial use but adds an attribution requirement for very large products. Terms can change per release.

**How much does the Kimi K3 API cost?**

There is no single price: each host sets its own per-token rates, and DeAI News had no verified K3 price as of 2026-08-20. Open-weight MoE APIs typically cost a fraction of frontier closed APIs. Compare providers' published pricing pages.

**What's the difference between Kimi K3 and K2.6?**

K3 is Moonshot's next generation at ~2.8T total and ~104B active parameters, as reported by Moonshot AI as of 2026-08-20. Migration is usually a model-ID swap on the same endpoint, but re-run your own evals — behavior and refusals can shift.

**What do you need to self-host Kimi K3?**

Roughly 2.8 TB of GPU memory at FP8 (about double at BF16) for weights alone, plus KV cache — multi-node cluster territory with fast interconnect. vLLM and SGLang are the usual serving stacks. Most teams use a hosted API instead.

**What is Kimi K3's context length?**

Confirm the exact window on the model card — and on your provider's endpoint, since hosts sometimes cap context below the model maximum. Long prompts raise both latency and cost, so size your context budget before committing.

## Sources

- [Moonshot AI Platform](https://platform.moonshot.ai) — Moonshot AI
- [moonshotai organization (Kimi model cards)](https://huggingface.co/moonshotai) — Hugging Face
- [OpenRouter](https://openrouter.ai) — OpenRouter
- [Together AI](https://www.together.ai) — Together AI
- [Fireworks AI](https://fireworks.ai) — Fireworks AI
- [vLLM Documentation](https://docs.vllm.ai) — vLLM Project
- [SGLang](https://github.com/sgl-project/sglang) — SGLang Project
- [OpenAI Python SDK](https://github.com/openai/openai-python) — OpenAI
- [LMArena](https://lmarena.ai) — LMArena


---

# How to Run Llama 4 (Maverick & Scout) via API in 2026 — License Traps

*By DeAI Newsroom · 2026-08-24 · 8 min read*

Canonical: https://www.deai.org/news/run-llama-4-api

> **Summary:** Llama 4 Maverick and Scout are served by most major inference providers behind OpenAI-compatible endpoints, so integration is a base-URL swap.

You can run Llama 4 Maverick or Scout through almost any hosted inference provider by swapping the base URL and model ID of an OpenAI-compatible client; no Meta sign-up required. The real gate is legal: Meta's community license includes a 700 million monthly-active-user clause and an EU multimodal carve-out that decide whether you may build on it at all.

## Key takeaways

- Both Llama 4 models are mixture-of-experts with 17B active parameters per token: Maverick totals roughly 400B parameters across 128 experts; Scout totals roughly 109B across 16 experts.
- The 700M MAU clause forces companies above 700 million monthly active users to negotiate a separate license with Meta; everyone below that line uses the standard community license.
- The EU carve-out withholds rights to Llama 4's multimodal models from EU-domiciled builders, and both Maverick and Scout are natively multimodal, so text-only usage does not obviously escape it.
- Integration is provider-agnostic: nearly every Llama 4 host exposes an OpenAI-compatible `/v1/chat/completions` endpoint, so switching providers is a two-line change.
- Scout's model card lists a 10M-token context window, but most hosted providers serve far less. Verify the actual context limit before architecting around it.

## What are Llama 4 Maverick and Scout?

Meta released the first Llama 4 models in April 2025: two natively multimodal (image-plus-text in, text out) mixture-of-experts models. Maverick is the larger one (17B active parameters, roughly 400B total, 128 experts), built for general-purpose chat, coding, and image understanding. Scout (17B active, roughly 109B total, 16 experts) is the efficiency play: Meta says it was designed to fit on a single datacenter-class GPU, and its model card lists a 10M-token context length (Maverick's card lists 1M tokens).

Both are distributed as downloadable weights through llama.com and Hugging Face behind a license acceptance gate, and both are served as managed APIs by a long list of third-party providers. That second route is what this guide covers.

## The two Llama 4 license traps (and the fine print)

Llama 4 is open-weight, not open source. The distinction matters because Meta's community license attaches conditions that a true open-source license cannot. DeAI's explainer on [open-weight vs. open-source licensing](/open-weight-vs-open-source) covers the general picture; here are the Llama 4-specific traps.

### What is the Llama 4 license 700M MAU clause?

The Llama 4 Community License says, in substance: if the products or services made available by you or your affiliates had more than 700 million monthly active users in the month preceding the Llama 4 version release date, you must request a license from Meta, which Meta may grant or decline.

Practically, this touches only a handful of platform-scale companies. But two edge cases deserve attention:

- Affiliates count. If you are acquired by a company whose products exceed the threshold, expect the acquirer's counsel to re-examine your Llama 4 usage during due diligence.
- The test is pegged to the release date of the Llama version you use, per the license text, so read the current wording rather than assuming your growth trajectory is irrelevant.

For a typical startup or enterprise team, the clause is a non-issue. Document it, note it in your compliance file, and move on.

### What is the Llama 4 EU multimodal restriction?

This is the trap that actually bites. The license provides that, with respect to Llama 4's multimodal models, the usage rights are **not granted to individuals domiciled in, or companies with a principal place of business in, the European Union**. A carve-out within the carve-out exempts *end users* of a product or service that incorporates the models.

The practical reading, widely adopted since release:

- An EU consumer can use an app powered by Maverick. A company headquartered in the EU building that app is not licensed to use the model.
- Because *both* Maverick and Scout are multimodal models, restricting yourself to text-only prompts does not clearly remove you from scope. The restriction attaches to the model, not the modality you happen to call.

Meta has publicly pointed to European regulatory uncertainty as its rationale. Many EU-domiciled teams respond by choosing models without the carve-out rather than structuring around it. This is a legal judgment call, so involve counsel before you build a product on a model your entity may not be licensed to touch.

### Attribution, naming, and the acceptable use policy

Three more obligations builders routinely miss: products built on Llama 4 must display "Built with Llama"; derivative models you distribute must carry names beginning with "Llama"; and all use is subject to Meta's Acceptable Use Policy, which you must pass through to downstream users. None of these are onerous, but they are conditions of the license, not suggestions.

## How to run Llama 4 via API, step by step

### 1. Confirm your license position

Before writing code: under 700M MAU (including affiliates)? Not EU-domiciled (or have a legal opinion saying you're clear)? Comfortable with the attribution and AUP terms? If yes on all counts, proceed.

### 2. Choose a serving route

As of this writing (August 2026), Llama 4 Maverick and Scout are available across four categories of providers:

- Hyperscalers: AWS Bedrock, Azure AI Foundry, Google Cloud Vertex AI. Convenient if your data and compliance perimeter already lives there.
- Specialist inference clouds: Together AI, Fireworks AI, Groq, Deep Infra, SambaNova, Cerebras, Cloudflare Workers AI, and similar GPU-cloud APIs.
- Aggregators: OpenRouter and peers, which expose multiple backends behind one key and simplify failover.
- Decentralized networks: Morpheus, a decentralized inference marketplace, routes requests to independent operators and can be evaluated on the same criteria as any hosted provider.

Compare candidates on identical criteria: the exact model ID served, the context window actually exposed (frequently far below the model card's number), tool-calling and structured-output support, region availability, retention policy, and published pricing. Treat privacy claims (zero-retention promises, "operators can't see prompts") as policy statements by each provider, not independently verified guarantees. Output behavior also varies by serving stack; DeAI's refusal-index methodology scores how often a deployment declines benign prompts, and results will appear on the [model pages](/models) as they are published.

### 3. Get a key and find the exact model ID

Model identifiers differ per provider: one may use `meta-llama/Llama-4-Maverick-17B-128E-Instruct`, another a shorter slug. Copy the exact string from the provider's docs; a wrong ID is the most common cause of first-call 404s.

### 4. Make the call (OpenAI-compatible)

Python:

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://api.your-provider.example/v1",  # provider's endpoint
    api_key="YOUR_PROVIDER_KEY",
)

resp = client.chat.completions.create(
    model="meta-llama/Llama-4-Maverick-17B-128E-Instruct",  # exact ID varies by provider
    messages=[
        {"role": "system", "content": "You are a concise assistant."},
        {"role": "user", "content": "Summarize mixture-of-experts routing in two sentences."},
    ],
    temperature=0.3,
)

print(resp.choices[0].message.content)
```

curl:

```bash
curl https://api.your-provider.example/v1/chat/completions \
  -H "Authorization: Bearer $PROVIDER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "meta-llama/Llama-4-Scout-17B-16E-Instruct",
    "messages": [
      {"role": "user", "content": "Summarize mixture-of-experts routing in two sentences."}
    ]
  }'
```

For image input, use the standard OpenAI multimodal message shape (`image_url` content parts). Most Llama 4 hosts support it, but confirm per provider.

### 5. Build in portability

Because the interface is uniform, keep the base URL, key, and model ID in configuration. That gives you a same-day failover path to a second provider or an aggregator: cheap insurance against capacity crunches and deprecations.

## Should you self-host Llama 4 instead?

If you have steady, high-volume traffic or hard data-residency requirements, self-hosting is worth pricing out. Scout is the realistic candidate: Meta says it fits on a single datacenter-class GPU, and vLLM, SGLang, or TensorRT-LLM all serve it with an OpenAI-compatible front end. Maverick's roughly 400B total parameters demand a multi-GPU node and real operational maturity. Downloading weights from llama.com or Hugging Face means accepting the same community license; self-hosting changes your infrastructure, not your legal position.

## How much does the Llama 4 Maverick API cost?

There is no single answer, and any article quoting one number is quoting one provider's rate card on one day. What holds generally: hosted Llama 4 access is billed per million tokens, input tokens are usually priced below output tokens, and rates for open-weight models of this class are typically a fraction of frontier proprietary API pricing. Aggregator listings make cross-provider comparison easy, and provider pricing pages are the authoritative source. Check them at the time you commit, and watch for context-window tiers and batch discounts.

## FAQ

### What is the Llama 4 license 700M MAU clause?

If the products or services made available by you or your affiliates had more than 700 million monthly active users in the month before the Llama 4 version release date, you must request a separate license from Meta instead of using the standard community license. Below the threshold, it has no practical effect.

### What is the Llama 4 EU multimodal restriction?

The Llama 4 Community License does not grant rights to its multimodal models to individuals domiciled in, or companies with a principal place of business in, the EU. End users of products built on those models are exempt. Since both Maverick and Scout are multimodal, EU-domiciled builders should get legal advice before using either.

### How much does the Llama 4 Maverick API cost?

There is no single rate: each provider sets its own per-token pricing, typically a fraction of frontier proprietary API pricing. Compare current figures on provider pricing pages or an aggregator such as OpenRouter before committing.

### Is Llama 4 open source?

No. Llama 4 is open-weight: the weights are downloadable, but the license adds restrictions (the 700M MAU clause, the EU multimodal carve-out, attribution duties), so it fails the Open Source Initiative's definition of open source.

## FAQ

**What is the Llama 4 license 700M MAU clause?**

If the products or services made available by you or your affiliates had more than 700 million monthly active users in the month before the Llama 4 version release date, you must request a separate license from Meta instead of using the standard community license.

**What is the Llama 4 EU multimodal restriction?**

The Llama 4 Community License does not grant rights to its multimodal models to individuals domiciled in, or companies with a principal place of business in, the EU. End users of products built on those models are exempt from the restriction.

**How much does the Llama 4 Maverick API cost?**

There is no single rate: each provider sets its own per-token pricing, typically a fraction of frontier proprietary API pricing. Compare current figures on provider pricing pages or an aggregator such as OpenRouter before committing.

**Is Llama 4 open source?**

No. Llama 4 is open-weight: the weights are downloadable, but the license adds restrictions (the 700M MAU clause, the EU multimodal carve-out, attribution duties), so it fails the Open Source Initiative's definition of open source.

## Sources

- [Llama 4 Maverick 17B-128E-Instruct model card](https://huggingface.co/meta-llama/Llama-4-Maverick-17B-128E-Instruct) — Meta / Hugging Face
- [Llama 4 Scout 17B-16E-Instruct model card](https://huggingface.co/meta-llama/Llama-4-Scout-17B-16E-Instruct) — Meta / Hugging Face
- [Llama 4 — official site, downloads, and license](https://www.llama.com/llama4/) — Meta
- [llama-models repository (Llama 4 license text and model code)](https://github.com/meta-llama/llama-models) — Meta


---

# How to Run Qwen3.6 (27B & 35B-A3B) in 2026: Local, API, or Both

*By DeAI Newsroom · 2026-08-24 · 8 min read*

Canonical: https://www.deai.org/news/run-qwen-3-6

> **Summary:** Qwen3.6 comes in two self-hostable sizes: a 27B dense model that fits one high-VRAM GPU and a 35B-A3B mixture-of-experts. Run it locally with Ollama, LM Studio, or vLLM, or call it through any OpenAI-compatible provider — switching is a base-URL change.

Qwen3.6 is Alibaba's newest open-weight release, and the number that matters is 27: the 27B dense variant fits on a single high-VRAM GPU, so most builders can self-host it today. This guide walks through running it locally, calling it through an OpenAI-compatible API, and when a hybrid of both makes sense.

## Key takeaways

- Qwen3.6 ships in two sizes: a **27B dense** model and a **35B-A3B mixture-of-experts** that activates roughly 3B parameters per token.
- The 27B fits on **one high-VRAM GPU**; at 4-bit quantization, plan for weight memory in the mid-teens of gigabytes plus KV-cache overhead.
- Qwen3.6 is served through **OpenAI-compatible endpoints**, so switching providers (or moving from API to localhost) is a two-line config change.
- Against 397B-class flagships like Qwen3.5, Qwen3.6 trades peak capability for self-hostability and lower per-token serving cost.
- Most production teams land on a **hybrid**: local for steady or sensitive traffic, API for bursts and long-context overflow.

## What actually shipped: Qwen3.6 27B and 35B-A3B

The release comes in two flavors aimed at different serving profiles:

- **Qwen3.6 27B** is a dense transformer: every parameter participates in every token. Dense models are predictable to size and tune, which is why this is the variant most self-hosters will reach for first.
- **Qwen3.6 35B-A3B** is a mixture-of-experts (MoE) model. In Qwen's naming convention, "A3B" means roughly 3B parameters are active per token, routed from a 35B total pool. You get inference compute closer to a small model with a knowledge capacity closer to a large one, but all 35B weights still need to sit in memory.

On licensing: recent Qwen3-family releases have shipped under Apache 2.0, which permits commercial use, modification, and redistribution. Licensing can vary per model, so treat the model card and LICENSE file on the [Qwen organization page on Hugging Face](https://huggingface.co/Qwen) as the source of truth before you ship anything. We'll maintain spec sheets on our [model profiles](/models) as details settle.

## Can Qwen3.6 27B run on a single GPU?

Yes. That's the headline of this release. The 27B dense variant is sized to fit one high-VRAM GPU, and quantization widens the hardware pool considerably.

A planning heuristic practitioners use (a rule of thumb, not a measurement): at 4-bit quantization, weights occupy roughly half a gigabyte per billion parameters, putting a 27B model's weight footprint in the mid-teens of gigabytes. At 8-bit, that roughly doubles. On top of weights, budget for the KV cache, which grows with both context length and concurrent requests. Long-context sessions are usually what push a setup from "it fits" into out-of-memory territory.

The 35B-A3B has a similar memory story (all 35B weights must be resident), but because only ~3B parameters are active per token, it generates faster than a dense model of the same total size on identical hardware. If your GPU can hold either, the MoE is often the better latency pick.

Which quantization format depends on your serving stack: GGUF for llama.cpp, Ollama, and LM Studio; AWQ, GPTQ, or FP8 checkpoints for vLLM and SGLang. The model card on Hugging Face lists the official and community quant builds. CPU offload is possible in llama.cpp if you're short on VRAM, but expect a significant speed penalty. It's an evaluation tactic, not a production pattern.

## How do you run Qwen3.6 locally?

Three paths, in order of increasing operational commitment.

### Option 1: Ollama or llama.cpp (fastest start)

[Ollama](https://ollama.com) wraps llama.cpp in a one-command experience. Exact tags live on the model's page on ollama.com. Pull the one matching your VRAM budget:

```bash
ollama pull <qwen3.6-tag>   # check ollama.com for exact tags and quant sizes
ollama run <qwen3.6-tag>
```

Ollama automatically exposes an OpenAI-compatible endpoint on `http://localhost:11434/v1`, which matters for the hybrid pattern later. If you prefer to work closer to the metal, [llama.cpp](https://github.com/ggml-org/llama.cpp) gives you direct control over context size, GPU layers, and sampling.

### Option 2: LM Studio (GUI)

[LM Studio](https://lmstudio.ai) is the desktop route: search for Qwen3.6 in the built-in model browser, pick a GGUF quant, and toggle on the local server. It's the lowest-friction way to compare the 27B and 35B-A3B side by side before committing to a serving stack.

### Option 3: vLLM or SGLang (production serving)

For anything with concurrent users, use a throughput-oriented server. [vLLM](https://docs.vllm.ai) is the common default:

```bash
vllm serve <hf-org>/<qwen3.6-repo> --max-model-len 32768  # example; use the repo ID from the model card
```

This starts an OpenAI-compatible server on `http://localhost:8000/v1` with continuous batching and PagedAttention, the features that make a single GPU serve many users instead of one. SGLang is a peer worth evaluating on the same criteria.

## How do you call the Qwen3.6 API?

If you'd rather not manage GPUs, Qwen3.6's open weights mean multiple provider categories can host it: Alibaba's own hosted API, third-party serverless inference clouds, aggregators such as [OpenRouter](https://openrouter.ai) that route across backends, and Morpheus, a decentralized inference marketplace. We're not ranking them here. Apply identical criteria to each: the exact model ID on offer (27B vs 35B-A3B, and which quant), the published context window, the pricing page, rate limits, and the written data-retention policy. Note that retention claims like "zero logging" are policy statements by the provider, not independently verified facts. If privacy is load-bearing, local inference is the architecture where prompts genuinely never leave your hardware.

Virtually all of these expose the OpenAI chat-completions schema, so integration is a base-URL swap:

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://your-provider.example.com/v1",  # or http://localhost:8000/v1 for vLLM
    api_key="YOUR_API_KEY",                            # any string works for most local servers
)

resp = client.chat.completions.create(
    model="qwen3.6-27b",  # use the exact model ID your provider publishes
    messages=[{"role": "user", "content": "Explain mixture-of-experts in one paragraph."}],
)
print(resp.choices[0].message.content)
```

```bash
curl https://your-provider.example.com/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3.6-27b",
    "messages": [{"role": "user", "content": "Explain mixture-of-experts in one paragraph."}]
  }'
```

Because the same code path works against localhost and hosted endpoints, you can develop against one and fail over to the other without an SDK rewrite. Open-weight pricing at this size class is typically a fraction of frontier-API pricing, but check each provider's current pricing page rather than assuming.

## Qwen3.6 vs Qwen3.5 397B: what's the trade-off?

This is the comparison most upgraders are actually making. A 397B-class flagship like Qwen3.5 exists to maximize capability per response; Qwen3.6's 27B and 35B-A3B exist to maximize capability per dollar of infrastructure you control. The practical differences:

- **Deployment**: Qwen3.6 fits one GPU; a 397B-class model generally means multi-GPU serving or a hosted API.
- **Cost shape**: self-hosted Qwen3.6 is a fixed hardware cost with near-zero marginal tokens; the flagship is almost always a metered API bill.
- **Capability**: expect the larger model to hold an edge on hard reasoning, nuanced instruction following, and long-horizon tasks. How much edge is a question for independent evaluations and the official model cards, not something to take from launch posts, including this one.

For behavioral dimensions like refusal rates, DeAI's refusal-index methodology scores how often a model declines benign prompts across standardized categories; results are published on model profile pages as they complete.

## Local, API, or both: how do you decide?

**Run local** if you have steady request volume, data that can't leave your environment, or latency requirements that rule out a network hop. A single-GPU box running vLLM turns Qwen3.6 into a fixed-cost asset.

**Use the API** if your traffic is spiky, you're prototyping, or you occasionally need the longer context windows that hosted providers allocate more aggressively than your VRAM allows.

**Run both** is where most teams end up: route default traffic to your local endpoint, fail over to a hosted provider on local saturation or errors, and send sensitive workloads exclusively to local. Since both sides speak the same OpenAI schema, the router is a config file, not a rewrite. Our [self-hosting vs API cost breakdown](/self-hosting-vs-api-cost) walks through the breakeven math in detail.

The honest summary: Qwen3.6's 27B makes "own your inference" realistic for a much wider set of builders than the 397B generation did. Start with the API to validate your workload in an afternoon, then move steady traffic in-house once the usage curve justifies the GPU.

## FAQ

### Can Qwen3.6 27B run on a single GPU?

Yes. The 27B dense variant is sized to fit one high-VRAM GPU. At 4-bit quantization, plan for weight memory in the mid-teens of gigabytes plus KV-cache overhead; 8-bit roughly doubles the weight footprint.

### Is Qwen3.6 released under Apache 2.0?

Recent Qwen3-family releases have shipped under Apache 2.0, which permits commercial use and modification. Licensing can vary per model, so confirm on the model card and LICENSE file on Hugging Face before deploying.

### How does Qwen3.6 compare to Qwen3.5 397B?

Qwen3.6's 27B and 35B-A3B variants are far smaller: they self-host on a single GPU and cost less to serve per token. A 397B-class flagship targets maximum capability and generally needs multi-GPU serving or a hosted API.

### What's the difference between Qwen3.6 27B and 35B-A3B?

27B is dense: every parameter is active on every token. 35B-A3B is a mixture-of-experts with roughly 3B active parameters per token, so it runs faster at comparable quality but still needs all 35B weights in memory.

## FAQ

**Can Qwen3.6 27B run on a single GPU?**

Yes. The 27B dense variant is sized to fit one high-VRAM GPU. At 4-bit quantization, plan for weight memory in the mid-teens of gigabytes plus KV-cache overhead; 8-bit roughly doubles the weight footprint.

**Is Qwen3.6 released under Apache 2.0?**

Recent Qwen3-family releases have shipped under Apache 2.0, which permits commercial use and modification. Licensing can vary per model, so confirm on the model card and LICENSE file on Hugging Face before deploying.

**How does Qwen3.6 compare to Qwen3.5 397B?**

Qwen3.6's 27B and 35B-A3B variants are far smaller: they self-host on a single GPU and cost less to serve per token. A 397B-class flagship targets maximum capability and generally needs multi-GPU serving or a hosted API.

**What's the difference between Qwen3.6 27B and 35B-A3B?**

27B is dense — every parameter is active on every token. 35B-A3B is a mixture-of-experts with roughly 3B active parameters per token, so it runs faster at comparable quality but still needs all 35B weights in memory.

## Sources

- [Qwen models on Hugging Face](https://huggingface.co/Qwen) — Hugging Face
- [Qwen documentation and blog](https://qwenlm.github.io) — Qwen Team
- [vLLM documentation](https://docs.vllm.ai) — vLLM Project
- [llama.cpp](https://github.com/ggml-org/llama.cpp) — ggml-org
- [Ollama](https://ollama.com) — Ollama
- [LM Studio](https://lmstudio.ai) — LM Studio
- [OpenRouter](https://openrouter.ai) — OpenRouter


---

# How to Run Qwen3 Coder 480B via API in 2026 (Apache 2.0)

*By DeAI Newsroom · 2026-08-24 · 8 min read*

Canonical: https://www.deai.org/news/run-qwen3-coder-api

> **Summary:** Qwen3 Coder 480B-A35B is an Apache 2.0 mixture-of-experts coding model you can call from almost any OpenAI-compatible provider by swapping a base URL and API key.

Qwen3 Coder 480B-A35B is an Apache 2.0 coding model you can call through almost any OpenAI-compatible API: get a key, swap the base URL, and set the model ID. This guide covers providers, working Python and curl code, agent setup, and costs for the 480-billion-parameter mixture-of-experts model.

## Key takeaways

- Qwen3 Coder 480B-A35B is a mixture-of-experts model with 480B total parameters but only 35B active per token, released under Apache 2.0, so commercial use is permitted.
- The model card lists a 256K-token native context window, extendable to 1M tokens with YaRN, which is why coding agents use it for whole-repo tasks.
- Any OpenAI-compatible provider works: the integration is a base-URL swap plus an API key, roughly five lines of Python.
- The FP8 checkpoint alone is about half a terabyte, so self-hosting means an 8-GPU node; API access bills per token with no infrastructure to run.
- Pricing varies by provider and is typically a fraction of frontier closed-API pricing. Compare current rates on each provider's pricing page or DeAI's [model catalog](/models).

## What is Qwen3 Coder 480B, and why do coding agents use it?

Qwen3 Coder 480B-A35B-Instruct is Alibaba's flagship open-weight coding model. Three properties explain its adoption in agent tooling:

**Architecture.** It is a mixture-of-experts model: 480 billion parameters total, but only about 35 billion are active for any given token. That gives it the capacity of a very large model with inference costs closer to a mid-sized one, and that's the main reason providers can serve it at open-weight prices.

**Context.** According to the [Hugging Face model card](https://huggingface.co/Qwen/Qwen3-Coder-480B-A35B-Instruct), it supports 262,144 tokens natively, extendable to roughly 1M tokens with YaRN extrapolation. Agentic coding workloads (multi-file refactors, long tool-call transcripts, repo-scale search) are the ones that choke on small context windows.

**License.** The weights ship under [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0). You can use the model commercially, fine-tune it, and redistribute derivatives, with no copyleft and no usage restrictions beyond attribution. For teams building products on top of a coding model, that removes the license-review friction that comes with custom community licenses.

The Qwen team reports strong results on agentic-coding evaluations. Treat those as vendor-reported claims: benchmark scores transfer imperfectly to real codebases, so validate on your own tasks before committing.

## How much does Qwen3 Coder 480B A35B cost via API?

There is no single price. Every provider sets its own per-million-token rates, and they change often enough that any number printed here would be stale. A few structural points hold regardless:

- **Input and output tokens are billed separately**, with output tokens costing more. Agent loops are input-heavy: every turn resends the conversation history, so prompt caching (where offered) matters more than the headline rate.
- **Precision tiers exist.** Some providers serve the FP8 checkpoint at a lower price than full-precision weights. For most coding tasks the quality difference is small; test both if your provider exposes them.
- **Open-weight models typically cost a fraction of frontier closed APIs** for comparable coding tasks, because multiple providers compete on the same weights.

The practical move: estimate your tokens per task (run a representative agent session and count), then compare providers on dollars per completed task rather than dollars per million tokens. Check each provider's published pricing page for current rates, and DeAI's [/models](/models) catalog for a side-by-side view of who serves what.

## Where can you get a Qwen3 Coder API key?

Model availability shifts, so confirm the exact model ID in each provider's catalog before wiring anything up. The main routes:

**First-party hosting.** [Alibaba Cloud Model Studio](https://www.alibabacloud.com/en/product/modelstudio) (DashScope) serves the Qwen family directly and offers an OpenAI-compatible endpoint mode. This is the canonical source and usually carries the full-precision variant.

**Aggregators.** [OpenRouter](https://openrouter.ai/models) fronts multiple upstream providers behind one key and one API, so you can compare price, latency, and uptime per upstream and fail over between them.

**Serverless inference clouds.** Providers such as Together, Fireworks, and Novita have historically carried the Qwen3 Coder family at launch. Each exposes an OpenAI-compatible endpoint; check their model lists for the 480B variant.

**Decentralized marketplaces.** A decentralized inference marketplace such as Morpheus routes requests to independent GPU operators rather than a single company's datacenter. The integration surface is the same: an OpenAI-compatible endpoint, a key, and a model ID. So are the evaluation criteria: price, latency, context length supported, and retention terms.

On privacy: any provider's "we don't retain prompts" language is a policy statement, not a verified fact, regardless of whether the provider is centralized or decentralized. If you are sending proprietary code, read the retention terms and prefer providers whose policies you have actually reviewed.

## Quickstart: call the Qwen3 Coder API with an OpenAI-compatible client

Every provider above speaks the OpenAI chat-completions schema. The entire integration is a base URL, a key, and a model ID. The exact model string varies by provider, so copy it from their catalog.

### Python

```python
import os
from openai import OpenAI

client = OpenAI(
    base_url=os.environ["PROVIDER_BASE_URL"],  # e.g. https://api.example.com/v1
    api_key=os.environ["PROVIDER_API_KEY"],
)

response = client.chat.completions.create(
    model="Qwen3-Coder-480B-A35B-Instruct",  # exact ID varies by provider
    messages=[
        {"role": "system", "content": "You are a senior software engineer."},
        {"role": "user", "content": "Write a Python async function that retries an HTTP request with exponential backoff and jitter."},
    ],
    temperature=0.2,
    max_tokens=2048,
)

print(response.choices[0].message.content)
```

### curl

```bash
curl "$PROVIDER_BASE_URL/chat/completions" \
  -H "Authorization: Bearer $PROVIDER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen3-Coder-480B-A35B-Instruct",
    "messages": [
      {"role": "user", "content": "Refactor this function to be iterative instead of recursive."}
    ],
    "temperature": 0.2,
    "stream": true
  }'
```

Two practical notes. First, keep `temperature` low (0–0.3) for code generation; the model is well-behaved at low sampling temperatures and you want determinism. Second, always enable streaming for agent workloads. Long generations over a non-streaming connection are how you hit spurious timeouts.

## How do you wire Qwen3 Coder into a coding agent?

Most agent harnesses (CLI coding assistants, IDE plugins, custom loops) accept an OpenAI-compatible endpoint through environment variables:

```bash
export OPENAI_BASE_URL="$PROVIDER_BASE_URL"
export OPENAI_API_KEY="$PROVIDER_API_KEY"
export AGENT_MODEL="Qwen3-Coder-480B-A35B-Instruct"
```

The Qwen team maintains Qwen Code, an open-source coding CLI built for this model family, with setup instructions in the [Qwen3-Coder GitHub repository](https://github.com/QwenLM/Qwen3-Coder). Other harnesses work too, since the model was trained for agentic tool use: function calling follows the standard OpenAI `tools` schema on every provider listed above.

Three things to verify before trusting it in a loop:

1. **Tool-call formatting.** Providers occasionally mangle the `tool_calls` field on streaming responses. Run one full tool-use round trip before scaling up.
2. **Context accounting.** A 256K window fills faster than you think when the agent pastes file contents. Track cumulative input tokens per session; they drive both cost and truncation risk.
3. **Refusal behavior.** Some providers layer moderation that refuses benign security-adjacent coding tasks. DeAI's refusal-index methodology is designed to score exactly this; until public results ship, test your own edge cases (exploit-writeups, fuzzers, auth code) against each provider you shortlist.

## Should you self-host instead?

The math is unforgiving but simple. FP8 weights for 480B parameters occupy roughly 480GB; BF16 doubles that to nearly a terabyte. Add KV cache for long contexts and you are looking at a single 8-GPU node (H100/H200 class) as the realistic minimum for the FP8 variant. With that hardware, [vLLM](https://docs.vllm.ai) or SGLang serves an OpenAI-compatible endpoint in one command:

```bash
vllm serve Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8 \
  --tensor-parallel-size 8 \
  --max-model-len 262144
```

Self-hosting wins when you have sustained volume (an always-on node beats per-token billing only if you keep it busy), strict data-residency requirements, or fine-tuning plans. APIs win on everything else: no capacity planning, no on-call, and you can switch providers by changing one environment variable. If you want local inference without the datacenter, the smaller Qwen3 Coder 30B-A3B variant runs on a single high-end GPU and covers lighter agent tasks.

## What is the best coding open model API?

Honest answer: it depends on your workload, and anyone claiming a single winner is selling something. The comparison criteria that actually matter are tool-calling reliability on your harness, effective context length as served (some providers cap it below the model's native window), latency at your concurrency, price per completed task, and retention policy. Qwen3 Coder 480B is a strong default because of its Apache 2.0 license and wide provider availability, but the open-weight coding field moves fast. For a maintained cross-provider comparison, see DeAI's guide to the [best LLM API for AI agents](/best-llm-api-for-ai-agents) and the [/models](/models) catalog.

## FAQ

### How much does the Qwen3 Coder 480B A35B API cost?

Providers bill per million tokens and rates vary, so check each provider's pricing page. Open-weight models like this typically cost a fraction of frontier closed APIs. Self-hosting only undercuts APIs at sustained high volume.

### What is the best coding open model API?

There is no single winner. Compare on tool-calling reliability, context length, latency, and price. Qwen3 Coder 480B is a leading Apache 2.0 option; DeAI's [best LLM API for AI agents](/best-llm-api-for-ai-agents) guide compares the field.

### Is Qwen3 Coder 480B licensed for commercial use?

Yes. The weights are released under Apache 2.0, which permits commercial use, modification, and redistribution with attribution and no copyleft. Confirm the terms on the Hugging Face model card before shipping.

### Can I run Qwen3 Coder 480B locally?

Only with serious hardware: the FP8 checkpoint alone is roughly half a terabyte, so plan on an 8-GPU node. The smaller Qwen3 Coder 30B-A3B variant fits a single high-end GPU for local agent work.

### Does Qwen3 Coder support tool calling for agents?

Yes. It was trained for agentic coding with function calling, and OpenAI-compatible providers expose the standard tools schema. Test your harness end to end, because tool-call formatting differs across providers.

## FAQ

**How much does the Qwen3 Coder 480B A35B API cost?**

Providers bill per million tokens and rates vary, so check each provider's pricing page. Open-weight models like this typically cost a fraction of frontier closed APIs. Self-hosting only undercuts APIs at sustained high volume.

**What is the best coding open model API?**

There is no single winner. Compare on tool-calling reliability, context length, latency, and price. Qwen3 Coder 480B is a leading Apache 2.0 option; DeAI's best-llm-api-for-ai-agents guide compares the field.

**Is Qwen3 Coder 480B licensed for commercial use?**

Yes. The weights are released under Apache 2.0, which permits commercial use, modification, and redistribution with attribution and no copyleft. Confirm the terms on the Hugging Face model card before shipping.

**Can I run Qwen3 Coder 480B locally?**

Only with serious hardware: the FP8 checkpoint alone is roughly half a terabyte, so plan on an 8-GPU node. The smaller Qwen3 Coder 30B-A3B variant fits a single high-end GPU for local agent work.

**Does Qwen3 Coder support tool calling for agents?**

Yes. It was trained for agentic coding with function calling, and OpenAI-compatible providers expose the standard tools schema. Test your harness end to end, because tool-call formatting differs across providers.

## Sources

- [Qwen3-Coder-480B-A35B-Instruct model card](https://huggingface.co/Qwen/Qwen3-Coder-480B-A35B-Instruct) — Qwen / Hugging Face
- [Qwen3-Coder GitHub repository](https://github.com/QwenLM/Qwen3-Coder) — QwenLM
- [Apache License, Version 2.0](https://www.apache.org/licenses/LICENSE-2.0) — Apache Software Foundation
- [Alibaba Cloud Model Studio](https://www.alibabacloud.com/en/product/modelstudio) — Alibaba Cloud
- [OpenRouter model catalog](https://openrouter.ai/models) — OpenRouter
- [vLLM documentation](https://docs.vllm.ai) — vLLM Project


---

# Self-Hosting vs Inference APIs in 2026: The Real Cost Math

*By DeAI Newsroom · 2026-08-24 · 9 min read*

Canonical: https://www.deai.org/news/self-hosting-vs-api-cost

> **Summary:** Self-hosting beats API pricing only above a break-even token volume that depends on your utilization, not on sticker prices. Below that line, per-token APIs win; above it, a well-loaded GPU server usually does.

Self-hosting an LLM is cheaper than an inference API only when your monthly token volume clears one number: the break-even point where a GPU server's fixed cost, spread across your actual traffic, drops below per-token API pricing. Below that line, APIs win. Here is how to find your number.

## Key takeaways

- The whole decision reduces to one formula: break-even tokens per month = fixed monthly GPU cost ÷ (API price per token − your marginal serving cost per token).
- Utilization is the multiplier everyone forgets: a GPU running at 20% load costs 5× more per served token than the same GPU fully loaded.
- The serving software is free: vLLM, Ollama, and similar stacks are open source. The GPU, and the ops hours to keep it alive, are the real bill.
- In the illustrative worked example below, a hypothetical $2,000/month GPU node against a hypothetical $0.50-per-million-token API breaks even near 4.4 billion tokens per month, or roughly 1,700 tokens per second sustained around the clock.
- If your traffic is spiky, seasonal, or modest, a hosted endpoint (including private, no-retention options) usually beats owning silicon; the crossover comes later than most teams expect.

## Is self-hosting an LLM cheaper?

Sometimes, and the honest spread is wide. The two options have fundamentally different cost shapes, so the answer depends on where your workload sits on three axes: volume, utilization, and ops capacity.

An API is a pure variable cost: you pay per token, and when traffic stops, the bill stops. A self-hosted GPU is a pure fixed cost: the meter runs 24/7 whether you serve one token or one billion. Fixed costs beat variable costs only when you have enough volume to dilute them, which is why the entire debate collapses into a single computable threshold.

Three regimes emerge. Low or spiky volume almost always favors APIs. Sustained, predictable, high volume can favor self-hosting, sometimes decisively. And the crossover between them is arithmetic, not a vibe; the math is below.

## What an inference API actually bills you for

API pricing is per token, usually with separate rates for input and output tokens, and sometimes discounted rates for cached context. That simplicity hides how much is bundled in: redundancy, autoscaling, model upgrades, uptime engineering, and the on-call rotation are all inside the per-token price.

The most underrated line item is idleness. An idle API client costs nothing. An idle GPU costs exactly the same as a busy one. For workloads with diurnal traffic (business-hours copilots, bursty agents, weekend-quiet dev tools), this asymmetry dominates the math.

For current per-token rates across hosted providers, check our running roundup of the [cheapest LLM API options](/cheapest-llm-api), and verify against provider pricing pages directly, since both sides of this market reprice frequently.

## How much does a vLLM server cost?

vLLM itself is free, open-source serving software; the same is true of Ollama and most of the inference stack. The cost is everything around the software:

1. **The GPU instance.** This is the dominant line, typically the overwhelming majority of the monthly bill. You can rent on-demand (most expensive per hour, most flexible), reserve capacity with a commitment (cheaper, less flexible), or buy hardware and amortize it over its useful life while paying power and cooling.
2. **The host around the GPU.** CPU, RAM, fast NVMe storage for model weights, and bandwidth. Real money, but secondary.
3. **Ops time.** Driver updates, CUDA version wrangling, model migrations, throughput regressions, 3 a.m. outages. The line most spreadsheets omit entirely.
4. **Utilization.** A GPU is a fixed cost, so your cost per token is the fixed cost divided by tokens actually served. A half-loaded GPU doubles your per-token cost; a quarter-loaded one quadruples it.

That last point is where naive comparisons die. Sticker prices compare a fully-loaded GPU to an API; your traffic determines whether the GPU is ever fully loaded.

## GPU vs API cost calculator: the break-even formula

Define four variables:

- **F** is the fixed monthly cost of self-hosting: GPU instance plus supporting hardware plus an honest allowance for ops hours.
- **P** is the blended API price per token for your workload (weight input and output rates by your real traffic mix).
- **m** is the marginal cost per token when self-hosting: power and bandwidth. Usually small, but include it; leaving it out flatters the self-hosting case.
- **V** is your actual monthly token volume.

Then:

**Break-even volume B = F ÷ (P − m)**

If V sits comfortably above B, self-hosting is a cost candidate. If V is below B, the API is cheaper, full stop. A useful conversion: divide B by 2.592 million (the seconds in 30 days) to get the average tokens-per-second you must sustain around the clock to break even. That reframing is sobering: break-even volumes that sound abstract become very concrete when expressed as throughput you must generate at 4 a.m. on a Sunday.

## A worked example with clearly labeled assumptions

The numbers below are **illustrative placeholders, not market quotes**. Pull current GPU instance rates and per-token API prices from provider pricing pages and recompute; both move fast.

| Assumption | Illustrative value |
| --- | --- |
| Fixed monthly cost F (reserved single-GPU node serving a mid-size open-weight model) | $2,000 / month |
| Blended API price P | $0.50 per million tokens |
| Marginal self-host cost m | $0.05 per million tokens |
| Server capacity at your latency target | 5,000 tokens / second |

The math: B = $2,000 ÷ ($0.50 − $0.05) per million tokens ≈ **4.4 billion tokens per month**. That is roughly 148 million tokens per day, or about **1,700 tokens per second sustained 24/7**, around 34% utilization of the hypothetical 5,000 tok/s box.

Now the sensitivity, which matters more than the point estimate:

- If your blended API price doubles (heavier model mix, more output tokens), B halves to ~2.2 billion tokens.
- If you need high availability (and one GPU is one point of failure), F roughly doubles, and B doubles with it.
- If your traffic only keeps the box 15% loaded, your effective self-hosted cost per token is more than double the break-even assumption.

The levers are utilization, redundancy, and your real price mix, not the sticker price of the GPU.

## The hidden line items that break naive cost math

On the self-hosting side: idle capacity during off-hours; the second node you eventually need for failover; ops hours that scale with model churn, since open-weight releases arrive constantly and each upgrade costs evaluation and migration time; plus observability, storage, and egress.

On the API side, the hidden costs are different but real: rate limits biting at your peak, long-context workloads compounding per-token spend, and data leaving your perimeter, which for some teams is a compliance cost, not just a philosophical one. Note that hosted providers' zero-retention and no-training promises are policy statements by those providers, not independently verified facts; self-hosting remains the only architecture where prompt data verifiably never leaves hardware you control.

## The middle path: keep the privacy, drop the ops

The binary is a false one. Between "rent a GPU and run vLLM yourself" and "send everything to a frontier API" sits a growing tier: hosted endpoints for open-weight models, dedicated instances, and private endpoints with no-retention policies. Options include serverless open-weight hosts such as Together AI, Fireworks AI, and OpenRouter, as well as Morpheus, a decentralized inference marketplace that routes requests to independent operators. Evaluate them all on the same criteria: per-token price, model catalog, retention policy, and latency at your concurrency.

Because nearly everyone now exposes an OpenAI-compatible API, switching between self-hosted and hosted is a base-URL change, not a rewrite:

```python
import os
from openai import OpenAI

client = OpenAI(
    # Self-hosted vLLM: "http://localhost:8000/v1"
    # Hosted provider: the provider's OpenAI-compatible endpoint
    base_url=os.environ["LLM_BASE_URL"],
    api_key=os.environ.get("LLM_API_KEY", "local"),
)

resp = client.chat.completions.create(
    model=os.environ.get("LLM_MODEL", "your-model-name"),
    messages=[{"role": "user", "content": "Hello"}],
)
print(resp.choices[0].message.content)
```

```bash
export LLM_BASE_URL="http://localhost:8000/v1"   # or your provider's endpoint
export LLM_API_KEY="local"

curl "$LLM_BASE_URL/chat/completions" \
  -H "Authorization: Bearer $LLM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "your-model-name",
    "messages": [{"role": "user", "content": "Hello"}]
  }'
```

This is also the practical migration path for teams outgrowing a laptop: our guide to moving [from Ollama to a private endpoint](/ollama-to-private-endpoint) walks through keeping local-style privacy while shedding the ops burden. Many teams land on a hybrid: APIs and private endpoints for spiky traffic, self-hosted capacity for the steady baseline. That split is what the math usually recommends once you price both honestly.

## Which side of the line are you on?

Run the checklist:

- Compute B with real quotes. Is your sustained monthly volume comfortably above it, not just at peaks?
- Is your traffic smooth enough to keep a GPU loaded, or diurnal and bursty?
- Does data need to stay on hardware you control, full stop?
- Do you serve custom fine-tunes that hosted catalogs do not carry?
- Do you have the ops hours to own a GPU, honestly accounted?

If most answers point at volume, control, and customization, self-host. If they point at elasticity and simplicity, stay on APIs. If they split, hybrid; the base-URL swap makes that cheap.

## FAQ

### Is self-hosting an LLM cheaper than using an API?

Only above your break-even token volume: the monthly traffic at which a GPU server's fixed cost, spread across real usage, drops below per-token API pricing. Below that volume, APIs are cheaper because you never pay for idle capacity.

### How much does a vLLM server cost?

vLLM itself is free, open-source software. The real cost is the GPU instance (rented or amortized owned hardware), plus power, bandwidth, and ops time. Check current cloud GPU pricing pages for rates; they move frequently.

### How do I calculate GPU vs API cost?

Break-even tokens per month = fixed monthly GPU cost ÷ (API price per token − your marginal serving cost per token). Compare your real monthly volume against it; sustained volume well above that line favors self-hosting.

### When should you switch from an API to self-hosting?

When sustained volume sits comfortably above break-even, when data must stay on hardware you control, or when you serve custom fine-tunes. Spiky or modest traffic usually stays cheaper on APIs.

## FAQ

**Is self-hosting an LLM cheaper than using an API?**

Only above your break-even token volume — the monthly traffic at which a GPU server's fixed cost, spread across real usage, drops below per-token API pricing. Below that volume, APIs are cheaper because you never pay for idle capacity.

**How much does a vLLM server cost?**

vLLM itself is free, open-source software. The real cost is the GPU instance (rented or amortized owned hardware), plus power, bandwidth, and ops time. Check current cloud GPU pricing pages for rates — they move frequently.

**How do I calculate GPU vs API cost?**

Break-even tokens per month = fixed monthly GPU cost ÷ (API price per token − your marginal serving cost per token). Compare your real monthly volume against it; sustained volume well above that line favors self-hosting.

**When should you switch from an API to self-hosting?**

When sustained volume sits comfortably above break-even, when data must stay on hardware you control, or when you serve custom fine-tunes. Spiky or modest traffic usually stays cheaper on APIs.

## Sources

- [vLLM Documentation](https://docs.vllm.ai) — vLLM Project
- [Ollama](https://ollama.com) — Ollama
- [OpenAI API Pricing](https://openai.com/api/pricing/) — OpenAI
- [Models — Hugging Face](https://huggingface.co/models) — Hugging Face
- [NVIDIA H100 Tensor Core GPU](https://www.nvidia.com/en-us/data-center/h100/) — NVIDIA


---

# Switching From OpenRouter to a Direct Provider (2026): When and How

*By DeAI Newsroom · 2026-08-24 · 7 min read*

Canonical: https://www.deai.org/news/switch-from-openrouter

> **Summary:** Switching from OpenRouter to a direct provider is usually a base-URL and API-key swap, not a rewrite — the hard part is knowing when router economics stop making sense. The trigger is typically sustained volume on one or two models, plus a need for direct data terms.

Switching from OpenRouter to a direct provider usually comes down to one change: swapping a base URL and an API key. The harder question is when router economics stop making sense: typically once you serve one or two models at sustained volume. This guide covers the triggers, the migration steps, and the honest reasons to stay.

## Key takeaways

- The code change is typically 2 lines (`base_url` and `api_key`) because OpenRouter and most direct providers expose the same OpenAI-compatible chat-completions API.
- OpenRouter's core value is breadth: one key, one bill, and a catalog spanning hundreds of model endpoints. Going direct trades that convenience for one fewer intermediary and a direct commercial relationship.
- OpenRouter says it passes through upstream token pricing and charges a fee when you purchase credits; at sustained volume, that fee structure is worth modeling against direct pricing.
- Keeping OpenRouter configured as a fallback during migration gives you 2 layers of availability while you validate the new provider.
- If you call a long tail of models each month or you're still benchmarking, staying on OpenRouter is usually the correct decision.

## What you're actually paying OpenRouter for

OpenRouter is a routing and aggregation layer. You send one OpenAI-compatible request; it forwards that request to an upstream inference provider hosting the model, then returns the response. In exchange, you get a unified catalog, one API key, consolidated billing, provider fallbacks, and routing controls.

So what's the OpenRouter markup? OpenRouter says it charges the same per-token rates as the upstream providers and monetizes through a fee applied when you buy credits. It also offers a bring-your-own-key option for some providers, where you supply your own upstream credentials and OpenRouter acts purely as the router. Fee structures change, so treat the pricing page and docs as the source of truth rather than any article, including this one.

The practical implication: the "markup" isn't hidden in the token price; it's a visible fee layer plus the operational cost of having an intermediary in your request path. Whether that layer is worth it depends almost entirely on your usage shape.

## When does switching from OpenRouter to a direct provider make sense?

### You've converged on one or two models at sustained volume

Routers earn their keep when you need breadth. If your traffic has settled (say, one chat model and one embeddings or classification model running continuously), you're paying for aggregation you no longer use. At sustained volume, even a small per-dollar fee layer compounds into a line item worth eliminating, and direct providers may offer committed-use arrangements that a router can't pass through. For a framework on modeling that crossover, see our guide to finding the [cheapest LLM API](/cheapest-llm-api) for your workload shape.

### You need direct data-governance terms

Compliance teams often want a data-processing agreement with the entity that actually runs the GPUs. Through a router, your prompts transit an additional company's infrastructure, and retention is governed by a stack of policies: the router's plus each upstream provider's. Going direct shortens that chain to one counterparty and one set of terms to negotiate and audit.

### You want capacity guarantees and a support relationship

At meaningful scale, direct providers can discuss rate-limit headroom, dedicated capacity, and SLAs. A router can route around an outage, but it can't guarantee you throughput on a specific upstream deployment.

### Latency discipline matters

Every intermediary adds a network hop and a queueing point. Going direct removes one hop from the path. Whether that difference is material for your product is something only your own measurements can answer. Don't take anyone's marketing page as evidence, including ours.

## When should you stay on OpenRouter? (The honest list)

This is the part most migration guides skip. Stay on OpenRouter if:

- **You're still exploring.** If your model choice changes month to month, a router's catalog is the cheapest benchmarking infrastructure available.
- **You call a long tail of models.** Five, ten, or fifty models across different providers means five, ten, or fifty direct accounts, keys, bills, and quota dashboards. The aggregation fee is often cheaper than the engineering time.
- **You rely on automatic fallbacks.** If OpenRouter's provider failover is your availability story, going direct means rebuilding that yourself: health checks, retry logic, secondary providers.
- **Your volume is low or spiky.** Below a meaningful scale threshold, the fee layer is noise and the operational simplicity dominates.
- **You already use bring-your-own-key.** If you supply your own upstream keys, you've effectively already gone direct; OpenRouter is just your routing layer.

## How to switch: a six-step migration playbook

### Step 1: Audit 30 days of usage

Export or reconstruct your request logs. You need four numbers per model: request count, token volume, error rate, and latency distribution. Most teams discover their traffic is far more concentrated than they assumed, which either confirms the case for going direct or kills it.

### Step 2: Shortlist direct providers on identical criteria

Evaluate candidates on the same axes: exact model variant and quantization available, context window, throughput and rate limits, published pricing, data-retention terms, regions, and OpenAI-API compatibility. Direct providers fall into a few categories: first-party APIs from model developers, serverless inference clouds, and decentralized inference marketplaces such as Morpheus. Assess all of them against the same checklist. Our [OpenRouter alternatives](/openrouter-alternatives) roundup profiles the categories in more depth.

### Step 3: Make the code change

Because both sides speak the OpenAI-compatible API, the migration is usually configuration, not code. Python:

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://api.your-provider.example/v1",  # was https://openrouter.ai/api/v1
    api_key=os.environ["PROVIDER_API_KEY"],           # was OPENROUTER_API_KEY
)

resp = client.chat.completions.create(
    model="your-provider-model-id",  # check the provider's exact model naming
    messages=[{"role": "user", "content": "Hello"}],
)
```

And the equivalent curl check:

```bash
curl https://api.your-provider.example/v1/chat/completions \
  -H "Authorization: Bearer $PROVIDER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "your-provider-model-id",
    "messages": [{"role": "user", "content": "Hello"}]
  }'
```

### Step 4: Map model names and parameter quirks

Model IDs differ between OpenRouter's namespaced naming and each provider's own identifiers. Check the provider's model list and confirm the variant matches; the same open-weight model may be served at different quantizations or context lengths across providers. Also check how the parameters you depend on behave: `max_tokens`, `temperature`, streaming, tool calling. Any OpenRouter-specific routing options you were passing will need direct equivalents.

### Step 5: Run shadow traffic, then cut over

Before flipping production, replay a slice of real traffic against the direct endpoint and compare outputs, latency, and error rates against your OpenRouter baseline. Then shift a small percentage of live traffic, watch your application-level metrics, and ramp. This is your evaluation to run. No third-party benchmark substitutes for your own prompts on your own workload.

### Step 6: Keep a fallback and set a review date

Leave OpenRouter configured as a secondary route during the transition; it's a ready-made failover while you build confidence in the direct path. Set a calendar reminder 60 to 90 days out to compare actual spend and reliability against your projections. Migrations that looked right on paper sometimes get honestly reversed.

## The privacy question: OpenRouter's policy vs going direct

OpenRouter's privacy policy governs what it logs and retains as your prompts transit its infrastructure, and each upstream provider applies its own data terms on top. OpenRouter's docs also describe routing preferences that let you restrict which upstream providers may receive your traffic. Useful, but it means your data posture depends on a configuration layer you have to maintain.

Going direct collapses that to one policy: the provider's. That's simpler to audit, but apply the same skepticism everywhere — any provider's zero-retention or "we can't see your prompts" language is a policy statement, not an independently verified technical guarantee. If verifiable privacy is a hard requirement, that points toward architectures like local inference or providers with attestation mechanisms, which is a separate evaluation entirely.

## FAQ

### Does OpenRouter add a markup to model prices?

OpenRouter says it passes through upstream providers' token pricing and earns money through a fee on credit purchases. Terms can change, so confirm on its pricing page. Going direct removes that fee layer but adds per-provider billing overhead.

### What does switching from OpenRouter to a direct provider mean?

It means calling an inference provider's own API endpoint with its own API key instead of routing through OpenRouter. Because both typically expose an OpenAI-compatible API, the change is usually a base-URL and key swap, not a rewrite.

### How does OpenRouter handle prompt privacy?

OpenRouter is an intermediary: prompts transit its infrastructure on the way to upstream providers, and retention depends on its privacy policy plus each provider's data terms. Any provider's zero-retention language is a policy statement, not a verified guarantee.

### When should I stay on OpenRouter instead of going direct?

Stay if you're still exploring models, call a long tail of models each month, rely on its automatic fallbacks, or run low, spiky volume where per-token fees matter less than operational simplicity.

## FAQ

**Does OpenRouter add a markup to model prices?**

OpenRouter says it passes through upstream providers' token pricing and earns money through a fee on credit purchases. Terms can change, so confirm on its pricing page. Going direct removes that fee layer but adds per-provider billing overhead.

**What does switching from OpenRouter to a direct provider mean?**

It means calling an inference provider's own API endpoint with its own API key instead of routing through OpenRouter. Because both typically expose an OpenAI-compatible API, the change is usually a base-URL and key swap, not a rewrite.

**How does OpenRouter handle prompt privacy?**

OpenRouter is an intermediary: prompts transit its infrastructure on the way to upstream providers, and retention depends on its privacy policy plus each provider's data terms. Any provider's zero-retention language is a policy statement, not a verified guarantee.

**When should I stay on OpenRouter instead of going direct?**

Stay if you're still exploring models, call a long tail of models each month, rely on its automatic fallbacks, or run low, spiky volume where per-token fees matter less than operational simplicity.

## Sources

- [OpenRouter Documentation](https://openrouter.ai/docs) — OpenRouter
- [OpenRouter Privacy Policy](https://openrouter.ai/privacy) — OpenRouter
- [OpenRouter Model Catalog](https://openrouter.ai/models) — OpenRouter
- [OpenAI API Reference](https://platform.openai.com/docs/api-reference) — OpenAI
- [Hugging Face Inference Providers Documentation](https://huggingface.co/docs/inference-providers) — Hugging Face


---

# Together AI vs Fireworks AI (2026): Speed, Price, and Privacy Compared

*By DeAI Newsroom · 2026-08-24 · 9 min read*

Canonical: https://www.deai.org/news/together-vs-fireworks

> **Summary:** Together AI and Fireworks AI were both founded in 2022 and sell the same core product: fast, OpenAI-compatible serving of open-weight models, billed per million tokens.

Together AI and Fireworks AI are closer than their marketing suggests: both were founded in 2022, both serve the same open-weight models through OpenAI-compatible APIs, and both bill per million tokens. The real decision comes down to 4 axes: speed on your model, price at your volume, retention terms, and fine-tuning needs.

## Key takeaways

- Both companies were founded in 2022 and both expose OpenAI-compatible endpoints. Switching between them is one base URL and one API key, so a real A/B test costs you an afternoon, not a sprint.
- Neither has a durable speed advantage. Latency varies by model, quantization, region, and time of day; the only numbers that matter are time-to-first-token and tokens/second measured on your own prompts.
- Both bill per million tokens on serverless tiers and per GPU capacity on dedicated deployments, typically at a fraction of frontier closed-API pricing, but rate cards change monthly, so verify on the providers' pricing pages or [DeAI's price tracker](/prices) before budgeting.
- Privacy at both is a policy commitment, not an architecture. Prompts leave your infrastructure and are processed on their GPUs; retention and training-use terms are self-published statements, not verified facts.
- Fine-tuning is a genuine differentiator: Together AI offers full and LoRA fine-tuning, while Fireworks AI emphasizes LoRA adapters with multi-adapter serving. Check current docs for the exact workflow you need.

## Together AI vs Fireworks AI at a glance

| | Together AI | Fireworks AI |
|---|---|---|
| Founded | 2022 | 2022 |
| Background | Research-heavy team; publishes inference research | Founded by former Meta PyTorch engineers |
| API | OpenAI-compatible | OpenAI-compatible |
| Billing units | Per-token serverless; per-GPU dedicated | Per-token serverless; per-GPU dedicated |
| Fine-tuning | Full fine-tuning and LoRA | LoRA-focused, multi-adapter serving |
| Marketed strength | Broad model catalog, serving research | Custom serving stack, low latency |

The table is deliberately symmetric because the products are. The differences that will actually affect your bill and your p95 latency live one level down.

## What each company actually sells

### Together AI in one paragraph

Together AI sells hosted inference for open-weight models (the Llama, Qwen, DeepSeek, and Mistral families and a long tail beyond them) through serverless, pay-per-token endpoints, plus dedicated endpoints for teams that need guaranteed capacity. It also sells fine-tuning (both LoRA and full fine-tuning) and raw GPU clusters for teams that want to run their own serving stack. The company markets a research-driven approach to serving optimization and publishes inference research alongside the product.

### Fireworks AI in one paragraph

Fireworks AI sells the same core thing: serverless and dedicated serving of open-weight models through an OpenAI-compatible API. The founding team built PyTorch infrastructure at Meta, and the company markets a custom serving stack, rather than off-the-shelf open-source serving, as its latency edge. It offers LoRA-based fine-tuning with multi-adapter serving, structured output, and function-calling support. Like Together, it positions itself as the fast, cheaper alternative to frontier closed APIs.

## Which is faster: Together AI or Fireworks AI?

Both providers market speed aggressively, and both publish latency claims. Treat all of those as claims. There is no permanent winner here, for structural reasons:

- **Quantization differs.** The same model served at FP8 versus BF16 behaves differently on both speed and output quality. Providers don't always disclose which quant a given endpoint runs.
- **Load varies.** Serverless endpoints are shared infrastructure. A benchmark taken at 10 a.m. on a Tuesday says little about your p95 during a traffic spike.
- **Your prompt shape matters.** Time-to-first-token scales with input length; throughput matters more for long outputs. A provider can win one and lose the other on the same model.

Third-party trackers such as Artificial Analysis publish cross-provider latency snapshots, which are useful for shortlisting but age quickly as providers retune their stacks. The defensible method is to measure both providers yourself, on the same model, with your production prompts, at your expected concurrency, at three different times of day. Record time-to-first-token, tokens per second, and error rates. Also run a small eval set against both endpoints: silent quantization or serving changes can shift model behavior, not just speed. (DeAI's refusal-index methodology scores exactly this class of provider-side behavior drift against reference model behavior; results are published as they complete.)

## How does Together AI pricing compare to Fireworks AI pricing?

Both providers publish per-million-token rate cards tiered by model size, and both bill dedicated deployments by GPU capacity, typically with discounts for committed use. Input and output tokens are usually priced differently, with output tokens costing more. Both sit well below frontier closed-API pricing for comparable open-weight models.

Because rate cards change frequently (providers cut prices and re-tier models regularly), this article deliberately quotes no specific numbers. DeAI had no verified snapshot of either rate card as of 2026-08-20, and any figure printed here would be stale within weeks. Check both pricing pages directly, or the cross-provider tables on [DeAI's /prices page](/prices), on the day you budget.

What actually determines your bill, beyond the headline rate:

- **Output-token share.** A chatbot with long answers costs more per request than a classifier with short ones, at the same rate card.
- **Context length.** Long-context tiers are often priced separately. If your RAG pipeline stuffs 32K-token prompts, price that tier, not the headline one.
- **Serverless vs dedicated crossover.** At sustained volume, dedicated GPUs can beat per-token pricing; at spiky volume, they can't. Model a month of real traffic both ways.
- **Fine-tuning hosting.** Training is usually cheap; hosting the resulting adapter or model is the recurring cost. Price the full lifecycle.

## Which provider is more private?

Neither is private by architecture. Both are centralized US cloud providers: your prompts leave your machines, cross the internet, and are processed on GPUs they operate. TLS protects traffic in transit; nothing protects prompts from the provider itself except the provider's own policies.

Both companies publish terms covering data retention and whether customer data is used for training, and enterprise tiers at both may include contractual zero-retention commitments. Read the current versions of those documents before you sign anything, and understand what they are: policy statements made by the provider, not independently verified facts and not technical guarantees. If a provider says it doesn't retain prompts, you are trusting the statement, not verifying it.

For workloads where that trust model is unacceptable, such as legal, medical, sensitive code, or regulated data, the honest options are self-hosting open weights on your own hardware or in your own cloud tenancy (vLLM and SGLang are the standard open-source serving stacks), or evaluating providers whose architecture, not just their policy, minimizes data exposure. Neither Together AI nor Fireworks AI is architecturally zero-knowledge, and neither claims to be.

## Fine-tuning, dedicated deployments, and the catalog long tail

This is where the two genuinely diverge on paper:

- **Fine-tuning depth.** Together AI offers both LoRA and full fine-tuning, which matters if you're adapting a model's behavior substantially rather than adding a narrow skill. Fireworks AI focuses on LoRA adapters and markets the ability to serve many adapters efficiently from shared infrastructure, which is attractive if you run per-customer or per-task adapters.
- **Dedicated capacity.** Both offer dedicated deployments for consistent latency and isolation. Together also rents raw GPU clusters, which suits teams running their own serving stack. Fireworks offers on-demand and reserved dedicated tiers.
- **Catalog coverage.** Both serve the major open-weight families (Llama, Qwen, DeepSeek, Mistral), but the long tail differs week to week: specific model variants, quantizations, and context-length tiers. If your production workload depends on a specific variant, confirm it's served, on both, before benchmarking anything else.

## How to benchmark both in one afternoon

Because both APIs are OpenAI-compatible, the test harness is one client with two configurations:

```python
import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["PROVIDER_API_KEY"],
    base_url=os.environ["PROVIDER_BASE_URL"],
    # Together AI:  https://api.together.xyz/v1
    # Fireworks AI: https://api.fireworks.ai/inference/v1
)

response = client.chat.completions.create(
    model=os.environ["MODEL_NAME"],  # use the identical model on both
    messages=[{"role": "user", "content": "Summarize the attached contract clause."}],
    stream=True,  # stream so you can measure time-to-first-token
)
```

```bash
curl -X POST "$PROVIDER_BASE_URL/chat/completions" \
  -H "Authorization: Bearer $PROVIDER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "'"$MODEL_NAME"'",
    "messages": [{"role": "user", "content": "Summarize the attached contract clause."}],
    "stream": true
  }'
```

Run 50–200 real production prompts against each endpoint, at your expected concurrency, at three different times of day. Log TTFT, tokens/second, total latency, and failures. Then multiply the token counts from your logs by each provider's current rate card. That exercise (not any published benchmark, including ours) is the comparison that predicts your production experience.

## So which should you choose?

Choose on the axis that's actually binding for you. If you're latency-sensitive on one flagship model, benchmark both and weight TTFT. The winner will be workload-specific and may change next quarter. If you need full fine-tuning or raw GPU clusters, Together AI matches on paper. If your architecture is many LoRA adapters on shared serving, Fireworks AI matches on paper. If privacy is the top axis, neither is the right tool. Self-host.

The meta-answer, though, is that this is the wrong question to agonize over. Both APIs are OpenAI-compatible, both are month-to-month at the serverless tier, and switching costs are near zero. Build behind a thin provider-abstraction layer, run the afternoon benchmark, and keep the loser as your failover. For the wider field beyond these two, including providers with different privacy and decentralization trade-offs, see [our guide to the best open-source LLM APIs](/best-open-source-llm-api).

## FAQ

### How does Together AI pricing work?

Together AI charges per million tokens on serverless endpoints, with rates tiered by model size, and bills dedicated endpoints by GPU capacity. Rate cards change frequently, so confirm current numbers on Together's pricing page before budgeting.

### Is Fireworks AI good? A quick Fireworks AI review

Fireworks AI is an inference provider founded in 2022 by former Meta PyTorch engineers. It markets a custom serving stack focused on low latency, offers serverless and dedicated deployments plus LoRA fine-tuning, and exposes an OpenAI-compatible API.

### Which is faster, Together AI or Fireworks AI?

There is no durable public answer. Latency depends on the model, quantization, region, and live load, and both providers market speed heavily. Run identical prompts against both endpoints and compare time-to-first-token and tokens per second yourself.

### Which is more private, Together AI or Fireworks AI?

Both are centralized US cloud providers where prompts leave your infrastructure. Retention and training-use terms are self-published policy statements, not independently verified facts. For hard confidentiality requirements, self-host open weights.

### Can I switch between Together AI and Fireworks AI easily?

Yes. Both expose OpenAI-compatible chat-completions endpoints, so switching is typically a base-URL and API-key change. That makes an afternoon A/B benchmark cheap. Test both before committing to either.

## FAQ

**How does Together AI pricing work?**

Together AI charges per million tokens on serverless endpoints, with rates tiered by model size, and bills dedicated endpoints by GPU capacity. Rate cards change frequently, so confirm current numbers on Together's pricing page before budgeting.

**Is Fireworks AI good? A quick Fireworks AI review**

Fireworks AI is an inference provider founded in 2022 by former Meta PyTorch engineers. It markets a custom serving stack focused on low latency, offers serverless and dedicated deployments plus LoRA fine-tuning, and exposes an OpenAI-compatible API.

**Which is faster, Together AI or Fireworks AI?**

There is no durable public answer. Latency depends on the model, quantization, region, and live load, and both providers market speed heavily. Run identical prompts against both endpoints and compare time-to-first-token and tokens per second yourself.

**Which is more private, Together AI or Fireworks AI?**

Both are centralized US cloud providers where prompts leave your infrastructure. Retention and training-use terms are self-published policy statements, not independently verified facts. For hard confidentiality requirements, self-host open weights.

**Can I switch between Together AI and Fireworks AI easily?**

Yes. Both expose OpenAI-compatible chat-completions endpoints, so switching is typically a base-URL and API-key change. That makes an afternoon A/B benchmark cheap — test both before committing to either.

## Sources

- [Together AI](https://www.together.ai) — Together AI
- [Together AI Pricing](https://www.together.ai/pricing) — Together AI
- [Together AI Documentation](https://docs.together.ai) — Together AI
- [Fireworks AI](https://fireworks.ai) — Fireworks AI
- [Fireworks AI Pricing](https://fireworks.ai/pricing) — Fireworks AI
- [Fireworks AI Documentation](https://docs.fireworks.ai) — Fireworks AI
- [Artificial Analysis — Independent AI Benchmarks](https://artificialanalysis.ai) — Artificial Analysis
- [Meta Llama Models on Hugging Face](https://huggingface.co/meta-llama) — Hugging Face


---

# What Is Decentralized AI Inference? A Builder's Guide (2026)

*By DeAI Newsroom · 2026-08-24 · 8 min read*

Canonical: https://www.deai.org/news/what-is-decentralized-ai-inference

> **Summary:** Decentralized AI inference replaces a single cloud vendor with a protocol that routes requests across independent GPU operators serving open-weight models.

Decentralized AI inference means running open-weight models across a distributed network of independent GPU operators instead of one company's cloud. Your request is routed, executed, and paid for across three separate layers, so no single provider controls the model, the hardware, or the terms.

## Key takeaways

- Decentralized inference splits the serving stack into **3 layers**: routing, execution, and settlement. A protocol replaces one vendor's end-to-end control.
- Switching from a cloud API is often a **1-line change**: swap the OpenAI-compatible base URL and keep your existing SDK.
- You trade **1 trust assumption** (a vendor's policy) for **2 new ones**: the protocol's routing and settlement rules, plus the behavior of individual operators.
- Reliability is an engineering outcome, not a marketing claim: failover across **N operators** can beat a single endpoint, but only if your client actually retries.
- Ask **5 questions** before putting any network into production: models served, operator requirements, privacy enforcement, published pricing, exit path.

## What is decentralized AI inference?

Inference is the act of using a trained model to produce outputs: every chat completion, embedding, classification, or image generation call is inference. It is the steady-state cost of running AI, as opposed to training, which happens once per model version.

The default way to buy inference today is a cloud API: one company owns or rents the GPUs, loads the model weights, meters your usage, and sets the acceptable-use policy. You send a request; they send back tokens and an invoice.

Decentralized AI inference keeps that same request-response shape but replaces the single operator with a network of independent GPU operators, coordinated by a protocol or marketplace that handles discovery, metering, and payment. No one party decides which models are listed, what prompts are acceptable, or whether your account stays open.

The enabler is open-weight models. Because anyone can download weights like [Llama 3.1 8B](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct) and serve them with open-source tooling, the supply side of the market is permissionless. That is why these networks serve open weights almost exclusively, and why the model catalog question matters — a topic covered in the sibling explainer on [open-weight models](/what-are-open-weight-models). If your interest is the confidentiality angle specifically, see [what is private inference](/what-is-private-inference).

## How does decentralized inference work?

Architectures differ across networks, but nearly all of them decompose into the same three layers.

### 1. Routing and discovery

Your client talks to a gateway or router that exposes an [OpenAI-compatible API](https://platform.openai.com/docs/api-reference), the de facto interface standard for chat completions. The router maintains a live view of which operators are advertising which models, at what price and latency, and picks one for each request. Some networks put this routing logic on-chain; others run it as a gateway service. That is an architectural detail — the interface you code against usually looks identical either way.

### 2. Execution on someone else's GPU

The selected operator runs a standard open-source serving stack ([vLLM](https://docs.vllm.ai) is the most common, using continuous batching and paged attention to keep GPUs saturated) and streams tokens back to you over server-sent events. Because hardware is heterogeneous, latency varies more than on a single-vendor cloud. Some networks are experimenting with verification mechanisms (trusted execution environments, cryptographic receipts, spot-check recomputation) so you can confirm the requested model actually produced your output. Treat "verifiable inference" as active engineering, not a solved problem, unless the mechanism is documented.

### 3. Settlement

Every request is metered, usually per token, and the operator gets paid through the network's settlement layer. Payment rails vary: some networks settle in crypto tokens, others in conventional credits. From the builder's seat this is plumbing. What matters is that metering is done by the protocol rather than computed on a vendor's invoice you have to trust.

### The client-side change is usually one line

Because most networks expose an OpenAI-compatible surface, pointing an existing app at one looks like this:

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://<network-gateway>/v1",  # the network's OpenAI-compatible endpoint
    api_key="<your-key>",
)

resp = client.chat.completions.create(
    model="<model-name>",  # model identifiers vary by network
    messages=[{"role": "user", "content": "Hello"}],
)
print(resp.choices[0].message.content)
```

```bash
curl https://<network-gateway>/v1/chat/completions \
  -H "Authorization: Bearer <your-key>" \
  -H "Content-Type: application/json" \
  -d '{"model": "<model-name>", "messages": [{"role": "user", "content": "Hello"}]}'
```

The hard part is never the integration. It is everything in the next three sections.

## Who runs decentralized inference networks today?

The field is young and the architectures genuinely differ. A neutral sample, listed alphabetically:

- **Akash** — a marketplace for GPU compute where you deploy your own serving workload on leased hardware ([docs](https://akash.network/docs)).
- **Bittensor** — a network of incentivized subnets, several of which serve text generation, accessed via validators and gateways ([docs](https://docs.bittensor.com)).
- **GaiaNet** — a node network where individuals run model-serving nodes with their own knowledge and configuration ([docs](https://docs.gaianet.ai)).
- **io.net** — aggregated GPU supply from data centers and individuals, usable for compute jobs including inference ([docs](https://docs.io.net)).
- **Morpheus** — a decentralized inference marketplace where independent compute providers serve open-weight models behind a shared router.
- **Nosana** — a GPU grid where operators run containerized inference workloads ([docs](https://docs.nosana.io)).

This is a sample, not an endorsement or a ranking. One caution that applies to the whole category: networks publish dashboards with GPU counts, throughput, and usage figures, and those numbers are self-reported claims. Evaluate every network, including the ones above, on the identical criteria in the checklist below.

## Decentralized inference vs cloud API: what's the actual difference?

| Dimension | Cloud API | Decentralized network |
|---|---|---|
| Hardware operated by | One company | Many independent operators |
| Trust anchor | Vendor's terms of service | Protocol rules + operator behavior |
| Model catalog | Closed and open models | Open-weight models, almost always |
| Privacy basis | Vendor retention policy | Operator policy, unless a mechanism (e.g., TEEs) enforces more |
| Reliability model | A single SLA | Aggregate of many endpoints; client-side failover |
| Cost | Frontier API pricing | Typically a fraction of frontier-API pricing; check each network's published pricing page |
| Ops burden | Near zero | Retries, timeouts, fallback logic are on you |

Neither column is universally better. A cloud API gives you one throat to choke and one policy to read; a decentralized network means no single actor can quietly swap the model, change the terms, or close your account — but you inherit the engineering that the vendor used to do for you.

On privacy specifically: some networks and providers advertise zero-retention or "operators can't see your prompts" language. Unless that property is enforced by a technical mechanism (confidential computing, encrypted enclaves, documented attestation), it is a policy statement, not a verified fact. Ask which one you are getting.

## Is decentralized AI inference reliable?

Honestly: it varies, and it depends on how you connect.

A single-operator endpoint is roughly equivalent to self-hosting on someone else's box: nodes churn, models get reloaded, capacity appears and disappears. An aggregated router that fails over between operators is steadier, because one node's churn becomes a retry instead of an outage. Tail latency still varies more than a single-vendor cloud because the underlying hardware is heterogeneous. And capacity concentrates: small models are plentiful across networks, while very large models need scarce high-end GPUs, so availability thins out at the top of the size range.

The practical playbook is the same one you would use for any flaky upstream: retries with backoff, streaming with timeouts, pinned model versions, and a fallback provider sitting behind the same OpenAI-compatible interface so failing back is a config change, not a rewrite.

For current listings, DeAI News maintains a [tracker](/tracker) of networks and the models they serve. Separately, DeAI's refusal-index methodology scores endpoints against a fixed set of benign prompts to flag over-refusal behavior. The tracker is where those scores will appear as methodology runs complete.

## What should you check before switching?

Five questions, in order:

1. **Which models are actually served, and at what context lengths?** A network listing "Llama" may mean one quant at 8k context. Verify the exact variant your evals passed on.
2. **What does the network require of operators?** Stake, hardware minimums, attestation, or nothing at all; the answer tells you how much the operator set can be trusted to behave.
3. **How is privacy enforced — policy or mechanism?** "We don't log" is a promise. TEEs and attestation are engineering. Know which one protects your prompts.
4. **Where is the published pricing page, and how is metering done?** Per-token pricing should be public and the metering auditable, not computed in a black box.
5. **What is the exit path?** If the answer isn't "change one base URL back," you have lock-in wearing decentralization's clothes.

## FAQ

### How does decentralized inference work?

A client sends an OpenAI-compatible request to a router. The router selects a GPU operator running the requested open-weight model, the operator generates tokens with a standard serving engine such as vLLM, and a settlement layer pays the operator per request or per token.

### Is decentralized AI inference reliable?

It depends on the network and how you connect. Single-operator endpoints can churn; aggregated routers that fail over between operators are steadier. Latency and uptime vary, so production apps should add retries, timeouts, and a fallback provider regardless of network.

### Decentralized inference vs cloud API: what's the difference?

A cloud API is one company serving models from its own data centers under one policy. Decentralized inference routes requests across independent operators, so trust, privacy, and reliability depend on the protocol and operator set rather than a single vendor's terms.

### What models can you run on decentralized inference networks?

Almost always open-weight models from families like Llama, Qwen, Mistral, and DeepSeek, because operators must be able to download and serve the weights. Proprietary closed models are generally unavailable on these networks.

## FAQ

**How does decentralized inference work?**

A client sends an OpenAI-compatible request to a router. The router selects a GPU operator running the requested open-weight model, the operator generates tokens with a standard serving engine such as vLLM, and a settlement layer pays the operator per request or per token.

**Is decentralized AI inference reliable?**

It depends on the network and how you connect. Single-operator endpoints can churn; aggregated routers that fail over between operators are steadier. Latency and uptime vary, so production apps should add retries, timeouts, and a fallback provider regardless of network.

**Decentralized inference vs cloud API: what's the difference?**

A cloud API is one company serving models from its own data centers under one policy. Decentralized inference routes requests across independent operators, so trust, privacy, and reliability depend on the protocol and operator set rather than a single vendor's terms.

**What models can you run on decentralized inference networks?**

Almost always open-weight models — families like Llama, Qwen, Mistral, and DeepSeek — because operators must be able to download and serve the weights. Proprietary closed models are generally unavailable on these networks.

## Sources

- [OpenAI API Reference](https://platform.openai.com/docs/api-reference) — OpenAI
- [vLLM Documentation](https://docs.vllm.ai) — vLLM Project
- [Llama 3.1 8B Instruct Model Card](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct) — Meta (Hugging Face)
- [Akash Network Documentation](https://akash.network/docs) — Akash Network
- [Bittensor Documentation](https://docs.bittensor.com) — Opentensor Foundation
- [GaiaNet Documentation](https://docs.gaianet.ai) — GaiaNet
- [io.net Documentation](https://docs.io.net) — io.net
- [Nosana Documentation](https://docs.nosana.io) — Nosana


---

# What Does Zero Data Retention Actually Mean in AI APIs? (2026)

*By DeAI Newsroom · 2026-08-24 · 8 min read*

Canonical: https://www.deai.org/news/zero-data-retention-ai-api

> **Summary:** Zero data retention means a provider never persists your prompts or completions after serving a response — but it is a policy statement, not a verified fact, and coverage varies by tier, exceptions, and enforcement.

Zero data retention means an AI API provider does not store your prompts or completions after the response is delivered: nothing written to disk, nothing to subpoena, leak, or quietly train on later. But ZDR is a policy statement, not a verified fact, and what it covers varies widely. Four questions separate real commitments from marketing copy.

## Key takeaways

- Zero data retention (ZDR) is an absence-claim: a provider's written policy that your prompts and completions are never persisted. On a centralized API, you cannot independently verify it from the outside.
- Compare providers on 4 dimensions: **scope** (which data), **defaults** (which tiers), **exceptions** (abuse monitoring, legal holds), and **enforcement** (contract vs. architecture).
- On major hosted APIs, full ZDR is often gated behind enterprise tiers or per-request flags; standard tiers may retain prompts for a limited abuse-monitoring window. OpenAI's published policy describes up to 30 days.
- A decentralized inference marketplace removes the central server entirely, so there is no single operator database for logs to accumulate in. The individual node serving your request remains a residual trust point, though.
- DeAI's [Trust Tracker](/trust) scores provider data policies on identical criteria; pair it with our [guide to private AI APIs](/best-private-ai-api).

## What does zero data retention actually mean?

Every API request you send has a lifecycle, and retention can happen at several points along it:

1. **Transit.** Your prompt travels over TLS to the provider's edge. Load balancers and gateways see metadata (your IP address, timestamps, account ID, token counts) even when they never see plaintext.
2. **Inference.** The prompt is decrypted and loaded into the memory of a GPU worker. For the duration of the request it exists in plaintext in RAM and VRAM. On any conventional architecture this is unavoidable: the model cannot read what it cannot read.
3. **Response.** The completion streams back. Under a zero-retention policy, this is where the trail ends: nothing is written to a database, object store, or log file.
4. **Post-hoc.** Billing records, rate-limit counters, and sometimes the outputs of automated safety classifiers may persist even under ZDR, because providers carve them out as "metadata" or "safety logs."

So a precise definition: zero data retention means the provider does not write your prompts, completions, uploaded files, or embeddings to persistent storage after serving the response. It does not mean your data was never processed (it must be), and it rarely means *nothing at all* is kept. Billing and abuse-prevention records almost always survive.

When you read a ZDR claim, the first question is: zero retention of *what*, exactly?

## Does my AI provider keep my prompts?

It depends less on the brand and more on which tier, which product surface, and which policy document governs your account. The picture as of mid-2026, in qualitative terms:

**Frontier hosted APIs.** OpenAI, Anthropic, and Google all publish separate data terms for their API products versus their consumer chat apps. OpenAI's enterprise privacy documentation describes retaining API traffic for up to 30 days for abuse and safety monitoring under standard terms, with zero-retention arrangements available for qualifying customers and endpoints. Anthropic publishes its own safety-related retention language in its privacy and commercial terms. Google's Gemini API terms draw a well-known line between unpaid and paid tiers in how prompts may be handled. All three offer stronger contractual commitments, including no-training language, on enterprise agreements. These are published policies; read the current versions before relying on any summary, including this one.

**Open-weight inference clouds.** Providers such as Together, Fireworks, and similar GPU clouds run open-weight models and typically advertise that customer data is not used for training; retention specifics vary by provider and product tier. Aggregators add a wrinkle: when you call a model through OpenRouter, two parties' policies apply (the aggregator's and the downstream provider's), so "zero retention" has to hold at both hops.

**Privacy-positioned providers.** Some providers, such as Venice, build their product around a no-storage stance. Treat these the same as any other absence-claim: a policy statement, not a verified fact.

**Decentralized marketplaces.** Morpheus, a decentralized inference marketplace, takes a different structural approach: there is no central API server operated by one company that terminates all traffic, so there is no central log database to retain anything in the first place. The verification section below explains what that does and doesn't buy you.

## Is zero retention the same as "no training on your data"?

No. Conflating the two is the most common mistake buyers make. They are independent promises:

| Promise | What it commits to | What it does *not* commit to |
|---|---|---|
| No training on API data | Your prompts/completions are excluded from training pipelines | Deleting them; they may sit in logs for weeks |
| Zero data retention | Nothing is persisted after the response | Nothing was processed; metadata may persist |
| No human review | Staff don't read your prompts | No automated classifiers scanning them |

A provider can honor any one of these without the others. "We don't train on your data" is fully compatible with keeping complete prompt logs for a month. Read each commitment separately in the API terms.

## Which AI APIs don't store data?

The honest answer: verify at the policy level, not the brand level, and re-verify on a schedule. Providers revise their data terms regularly, sometimes with real substance. A practical screening process:

1. Find the API-specific data terms, not the consumer privacy policy, which often differs.
2. Check whether ZDR is the default on your tier, an enterprise add-on, or a per-request flag.
3. List the carve-outs: abuse monitoring, safety classifiers, legal holds, metadata.
4. Identify the enforcement mechanism: policy page, signed DPA, audit report, or architecture.

DeAI's [Trust Tracker](/trust) exists to make this comparison mechanical: it scores provider data policies on identical criteria so you can compare like with like. If you're shortlisting providers on privacy grounds, our [private AI API guide](/best-private-ai-api) walks the buyer-side criteria in detail.

One caution applies to every entry in any tracker, ours included: absence-claims are policy statements, not verified facts. A provider saying "we store nothing" is a commitment you can hold them to contractually. It is not something an outsider can observe directly.

## How can you verify a no-logging claim?

You can't, fully, from the outside of a centralized service. What you can do is climb a ladder of increasing assurance:

- **Level 1 — Published policy.** The weakest rung: self-reported and changeable, but it sets the baseline and creates a paper trail.
- **Level 2 — Contract.** A DPA, BAA, or enterprise addendum with explicit retention terms makes the promise legally enforceable. Still trust-based, but now with remedies.
- **Level 3 — Third-party audit.** SOC 2 Type II or ISO 27001 reports show an auditor examined controls. Note the limits: these audits assess whether stated controls operate effectively; they rarely attempt to prove a negative like "no logs exist anywhere."
- **Level 4 — Technical enforcement.** Confidential-computing deployments (TEEs) with remote attestation, or running open-weight models in your own VPC or on your own hardware, move retention from "their promise" to "your infrastructure." If you run the model, the retention question becomes an internal one.
- **Level 5 — Architectural removal.** In a decentralized inference marketplace, requests route to independent compute operators rather than to one company's API backend. There is no central server to retain your prompts: no unified log store, no single subpoena target, no one database to breach. The trust model shifts from "trust the company's policy" to "there is no company in the middle at all." The honest caveat: the individual operator serving your request sees your prompt in plaintext during inference and could, in principle, log it. What the architecture removes is the central retention point, not the theoretical possibility of a bad actor at the edge.

For compliance teams, ZDR also maps neatly onto GDPR's storage-limitation principle (Article 5): personal data kept no longer than necessary. A genuine zero-retention posture is the strongest form of that principle, if it's real.

## A checklist before you trust a zero-retention label

- Is ZDR the default on my tier, or an enterprise add-on / per-request flag?
- What is explicitly excluded: safety classifiers, abuse logs, billing metadata?
- Where is the retention window (if any) written down, and how long is it?
- Will the provider sign a DPA or BAA with retention terms spelled out?
- Is there a SOC 2 or equivalent report I can read under NDA?
- If it's a marketplace or aggregator: who are all the parties that see my prompt, and what does each one's policy say?
- If the answer matters enough: could I run an open-weight model in my own environment instead?

## FAQ

### Which AI APIs don't store data?

Several providers publish zero-retention policies, some by default and others only on enterprise tiers or via per-request flags. These are policy statements, not verified facts. DeAI's [Trust Tracker](/trust) compares current provider policies on identical criteria.

### Does my AI provider keep my prompts?

Check the API-specific data terms, not the consumer privacy policy. Many hosted APIs retain prompts for a limited window (OpenAI's published policy describes up to 30 days for abuse monitoring) unless your tier includes zero retention.

### What does zero data retention mean in an AI API?

The provider does not write your prompts, completions, files, or embeddings to persistent storage after returning the response. Data still exists transiently in memory during inference, and billing metadata or safety logs are often excluded.

### Is zero retention the same as not training on my data?

No. No-training means your data is excluded from model training; zero retention means it is not stored at all. A provider can honor one without the other. Evaluate both commitments separately in the API terms.

### How can you verify a no-logging claim?

You cannot fully verify from outside. Climb the assurance ladder: published policy, signed DPA/BAA, third-party audits, technical enforcement (TEEs or self-hosting), or architectures like a decentralized inference marketplace with no central server to retain logs.

## FAQ

**Which AI APIs don't store data?**

Several providers publish zero-retention policies — some by default, others only on enterprise tiers or via per-request flags. These are policy statements, not verified facts. DeAI's Trust Tracker (/trust) compares current provider policies on identical criteria.

**Does my AI provider keep my prompts?**

Check the API-specific data terms, not the consumer privacy policy. Many hosted APIs retain prompts for a limited window — OpenAI's published policy describes up to 30 days for abuse monitoring — unless your tier includes zero retention.

**What does zero data retention mean in an AI API?**

The provider does not write your prompts, completions, files, or embeddings to persistent storage after returning the response. Data still exists transiently in memory during inference, and billing metadata or safety logs are often excluded.

**Is zero retention the same as not training on my data?**

No. No-training means your data is excluded from model training; zero retention means it is not stored at all. A provider can honor one without the other — evaluate both commitments separately in the API terms.

**How can you verify a no-logging claim?**

You cannot fully verify from outside. Climb the assurance ladder: published policy, signed DPA/BAA, third-party audits, technical enforcement (TEEs or self-hosting), or architectures like a decentralized inference marketplace with no central server to retain logs.

## Sources

- [Enterprise privacy at OpenAI](https://openai.com/enterprise-privacy/) — OpenAI
- [Privacy Policy](https://www.anthropic.com/legal/privacy) — Anthropic
- [Gemini API Terms of Service](https://ai.google.dev/gemini-api/terms) — Google
- [Privacy Policy](https://openrouter.ai/privacy) — OpenRouter
- [Together AI](https://www.together.ai) — Together AI
- [Fireworks AI](https://fireworks.ai) — Fireworks AI
- [Venice](https://venice.ai) — Venice
- [Article 5: How to process personal data](https://gdpr.eu/article-5-how-to-process-personal-data/) — GDPR.eu


---

# 7 Zero-Retention AI APIs for Sensitive Workloads (2026)

*By DeAI Newsroom · 2026-08-24 · 9 min read*

Canonical: https://www.deai.org/news/zero-retention-ai-apis

> **Summary:** Seven hosted LLM APIs — OpenAI, Anthropic, Azure OpenAI, Amazon Bedrock, Google Vertex AI, Together AI, and Fireworks AI — publish no-training or zero-retention policies aimed at sensitive workloads.

If you're looking for an AI API with no data retention, the realistic shortlist is seven providers: OpenAI, Anthropic, Azure OpenAI, Amazon Bedrock, Google Vertex AI, Together AI, and Fireworks AI. The number that matters is 0: zero days of stored prompts. Only a signed contract, not a marketing page, makes that number enforceable.

## Key takeaways

- Seven hosted LLM APIs publish no-training-by-default policies, but true zero retention usually requires an approval process or an enterprise tier; it is rarely the default.
- "Won't train on your data" and "won't retain your data" are two different promises. Most providers keep limited abuse-monitoring logs (roughly 30 days in several published policies, as of 2026-08-20) unless you get that window waived.
- One signature decides HIPAA eligibility: without a Business Associate Agreement (BAA), a zero-retention API is still a non-starter for PHI.
- Zero providers' retention claims are externally verifiable. Treat "we don't store your prompts" as a policy statement and anchor it in a contract, a DPA, and audit scope.

## What does "zero data retention" actually mean?

Teams shopping for a private AI API routinely conflate four separate commitments. Untangling them is the whole game:

1. **No training.** The provider won't use your prompts or completions to improve its models. Every provider on this list publishes some version of this for API traffic.
2. **No retention.** The provider deletes (or never writes) prompt and completion content after the response is served. This is the "zero retention" claim, and it usually sits behind an approval process, an enterprise tier, or a specific endpoint configuration.
3. **Abuse-monitoring logs.** Most providers carve out an exception: content may be held briefly for trust-and-safety or abuse-prevention purposes. Several published policies describe a window of roughly 30 days (as of 2026-08-20); the whole point of a zero-retention agreement is collapsing that window to nothing.
4. **Transient processing.** Your prompt always exists in the provider's memory while the response is generated. "Zero retention" says nothing about this; it is a promise about storage *afterward*.

That last point is why DeAI's provider-trust methodology scores retention and privacy absence-claims ("operators can't see your prompts," "zero logs") as policy statements, not verified facts. You cannot audit a provider's disk from the outside. You can only move the claim from a web page into a contract.

## Which providers offer zero-retention APIs?

The matrix below summarizes each provider's *published* posture as of 2026-08-20. Terms change; confirm on the provider's current legal and documentation pages before signing anything.

| Provider | Trains on your data? | Default retention (published policy) | Path to zero retention | BAA for HIPAA? |
|---|---|---|---|---|
| OpenAI API | No (per policy) | Limited abuse-monitoring window | ZDR approval for eligible orgs/endpoints | Yes, for eligible customers |
| Anthropic API | No (per commercial terms) | Limited trust-and-safety retention | Enterprise arrangements | Yes, for eligible customers |
| Azure OpenAI | No | Abuse-monitoring storage unless modified | "Modified abuse monitoring" approval | Yes (Microsoft BAA) |
| Amazon Bedrock | No (docs: content not stored or used) | No provider-side content storage by default; you control your own logging | Default posture + keep your own invocation logging off | Yes (HIPAA-eligible service) |
| Google Vertex AI | No (without permission) | Limited caching/abuse logging per docs | Enterprise data-governance controls, residency options | Yes (Google Cloud BAA) |
| Together AI | Tier-dependent (per terms) | Varies by product tier | Enterprise / dedicated zero-retention terms | Enterprise; confirm |
| Fireworks AI | No (per policy) | Limited operational logs per policy | Enterprise zero-retention terms | Enterprise; confirm |

### Does the OpenAI API retain your prompts?

OpenAI's API terms state that customer inputs and outputs are not used for training. Its published policy describes a limited retention window for abuse monitoring, with a zero-data-retention (ZDR) option available to qualifying organizations on eligible endpoints after a review process. Practically: default API traffic is "no training, short retention," and ZDR is something you apply for, not something you toggle on.

### Is Anthropic's API zero-retention?

Anthropic's commercial terms commit to not training on customer content, with limited retention for trust-and-safety purposes described in its policies. Stricter arrangements are an enterprise conversation. Anthropic also operates a public trust center where you can review certifications and subprocessors before procurement.

### Is Azure OpenAI the enterprise HIPAA default?

Azure OpenAI is frequently the first shortlist entry for regulated workloads, for structural reasons rather than model quality: it sits inside Microsoft's compliance stack, Microsoft signs a BAA, and its documented "modified abuse monitoring" path lets approved customers remove prompt and completion storage entirely. If your organization already runs on Azure with a BAA in place, the marginal paperwork is small.

### How does Amazon Bedrock handle prompt data?

AWS's Bedrock documentation states that prompts and responses are not stored by the service to improve models, and that content logging (such as model invocation logging) is something you configure into *your* account, so the retention decision is largely yours to make and audit. Bedrock is a HIPAA-eligible service under the AWS BAA.

### What does Google Vertex AI promise?

Google's Vertex AI data-governance documentation commits to not using customer data to train foundation models without permission, and the platform offers data-residency controls, customer-managed encryption keys, and a BAA under Google Cloud's HIPAA commitments. The zero-retention story is assembled from platform controls rather than a single switch.

### Do independent inference clouds offer zero retention?

Together AI and Fireworks AI both serve popular open-weight models (Llama, Qwen, DeepSeek, and peers) behind OpenAI-compatible endpoints, and both publish no-training commitments with enterprise zero-retention or dedicated-deployment options. Terms vary by tier, so the operative question is what lands in your order form, not what the pricing page implies. These are the common picks when you want open-weight models without operating GPUs yourself.

### What about decentralized marketplaces?

A different trust model exists alongside hosted APIs. Morpheus, a decentralized inference marketplace, routes prompts to independent operators rather than to one provider's servers, so there is no single corporate retention policy to negotiate. That removes the central-logging question but moves verification to routing behavior and operator incentives, and any absence-of-logging claim is still a policy statement wherever your data lands. Evaluate it on the same criteria as the seven providers above: what is promised, by whom, and what is it anchored to.

## Is a zero-retention API enough for HIPAA?

No. HIPAA doesn't care about a retention marketing claim; it cares about a Business Associate Agreement. A provider can promise zero retention and still be unusable for PHI if it won't sign a BAA. Conversely, a signed BAA with a 30-day abuse log can be perfectly workable, because the BAA is what makes the handling lawful. The cloud platforms (Azure OpenAI, Bedrock, Vertex AI) are the common choices here precisely because BAAs are standard paperwork for them. Beyond the BAA, your own obligations remain: access controls, audit logging on your side, and minimum-necessary data in each prompt. HHS's covered-entity guidance is the right starting point if this is new territory.

## What makes an LLM API GDPR-compliant?

Nothing, by itself. No API is "GDPR certified," and compliance is shared. Your checklist as a controller: an Article 28 data-processing agreement with the provider, a lawful transfer mechanism (typically SCCs) if prompts leave the EEA, data-residency options where your risk assessment demands them, and a credible answer on erasure. That is exactly where retention policy matters, because you cannot honor a deletion request for data sitting in a vendor's abuse log. A zero-retention contract simplifies the erasure story considerably. Run a DPIA before sending personal data to any of these APIs.

## How do you verify a zero-retention claim before sending confidential data?

Since you can't inspect a provider's disks, verification is a procurement discipline:

1. **Get retention terms in the contract or DPA**, not the marketing page. If sales won't paper it, the policy doesn't exist for you.
2. **Ask for the abuse-monitoring window in writing**, and request the waiver or ZDR addendum explicitly.
3. **Check audit scope.** Does the SOC 2 or ISO 27001 report actually cover the API product you're calling?
4. **Review the subprocessor list.** Your prompt may touch more entities than the logo on the docs.
5. **Minimize client-side.** Redact PII, strip identifiers, and route through a gateway you control.
6. **Prefer private networking** (VPC endpoints, Private Link) so prompts don't traverse the public internet.
7. **Canary-test.** Send unique marker strings and exercise your DPA rights to see what the provider can produce.

DeAI maintains a [retention-policy matrix tracking each provider's current terms](/zero-data-retention-ai-api), and the [provider-trust methodology](/trust) scores exactly these claims: unverifiable absence-claims count as policy statements until they're anchored in contract.

## FAQ

### Which AI APIs offer zero data retention?

As of August 2026, seven major providers publish zero-retention or no-training policies: OpenAI, Anthropic, Azure OpenAI, Amazon Bedrock, Google Vertex AI, Together AI, and Fireworks AI. True zero retention usually requires approval or an enterprise tier. Get it in writing.

### Is there a HIPAA-compliant AI API?

Several providers sign BAAs, including Azure OpenAI, Amazon Bedrock, and Google Vertex AI, so their APIs can be used with PHI inside a compliant setup. Zero retention alone isn't enough: HIPAA requires a signed BAA plus your own administrative and technical safeguards.

### What makes an LLM API GDPR compliant?

No API is "GDPR certified." Compliance is shared: you need an Article 28 data-processing agreement, a lawful transfer mechanism (e.g., SCCs) for non-EU processing, data-residency options where required, and a plan for erasure requests that covers provider-side logs.

### Can I send confidential data to an AI API?

Yes, if the contract covers no training plus zero or minimal retention, and you minimize what you send (redaction, PII scrubbing). If the provider won't put retention terms in writing, move the workload to a private deployment or a self-hosted open-weight model.

### Does zero retention mean the provider never sees my prompt?

No. Your prompt is processed in the provider's systems to generate a response. Zero retention is a policy about what is stored afterward. It is a contractual promise you cannot verify from outside, so treat absence-claims as policy statements.

## FAQ

**Which AI APIs offer zero data retention?**

As of August 2026, seven major providers publish zero-retention or no-training policies: OpenAI, Anthropic, Azure OpenAI, Amazon Bedrock, Google Vertex AI, Together AI, and Fireworks AI. True zero retention usually requires approval or an enterprise tier — get it in writing.

**Is there a HIPAA-compliant AI API?**

Several providers sign BAAs — including Azure OpenAI, Amazon Bedrock, and Google Vertex AI — making their APIs usable with PHI inside a compliant setup. Zero retention alone isn't enough: HIPAA requires a signed BAA plus your own administrative and technical safeguards.

**What makes an LLM API GDPR compliant?**

No API is 'GDPR certified.' Compliance is shared: you need an Article 28 data-processing agreement, a lawful transfer mechanism (e.g., SCCs) for non-EU processing, data-residency options where required, and a plan for erasure requests that covers provider-side logs.

**Can I send confidential data to an AI API?**

Yes, if the contract covers no training plus zero or minimal retention, and you minimize what you send (redaction, PII scrubbing). If the provider won't put retention terms in writing, move the workload to a private deployment or a self-hosted open-weight model.

**Does zero retention mean the provider never sees my prompt?**

No. Your prompt is processed in the provider's systems to generate a response. Zero retention is a policy about what is stored afterward — a contractual promise you cannot verify from outside, so treat absence-claims as policy statements.

## Sources

- [OpenAI Enterprise Privacy](https://openai.com/enterprise-privacy/) — OpenAI
- [Anthropic Commercial Terms of Service](https://www.anthropic.com/legal/commercial-terms) — Anthropic
- [Data, privacy, and security for Azure OpenAI Service](https://learn.microsoft.com/en-us/legal/cognitive-services/openai/data-privacy) — Microsoft Learn
- [Data protection in Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/data-protection.html) — AWS Documentation
- [Data governance and generative AI](https://cloud.google.com/vertex-ai/generative-ai/docs/data-governance) — Google Cloud
- [Together AI Privacy Policy](https://www.together.ai/privacy) — Together AI
- [Fireworks AI](https://fireworks.ai) — Fireworks AI
- [HIPAA for Covered Entities](https://www.hhs.gov/hipaa/for-professionals/covered-entities/index.html) — U.S. Department of Health & Human Services
- [GDPR.eu — Complete guide to GDPR compliance](https://gdpr.eu/) — GDPR.eu

---

## Decentralized Inference Provider Tracker

Provider-level tracker of decentralized, private and open inference providers (17 providers). Every field is verification-graded — claim vs. independently-checked fact, never the reverse. Each provider has a page at https://www.deai.org/tracker/{id}, JSON at https://www.deai.org/api/v1/providers/{id}, and an append-only change feed at https://www.deai.org/api/v1/changes. Current state, one line per provider:

- Akash Network (GPU marketplace, tier 1), API live — record status: unverified — https://www.deai.org/tracker/akash
- Chutes (Bittensor subnet, tier 1), API live (OpenAI-compatible) — record status: unverified — https://www.deai.org/tracker/chutes
- Darkbloom (GPU marketplace, tier 1), API public-alpha (OpenAI-compatible) — record status: unverified — https://www.deai.org/tracker/darkbloom
- Dolphin Network (GPU marketplace, tier 1), API live — record status: unverified — https://www.deai.org/tracker/dolphin
- Engy (Bittensor subnet, tier 2), API live (OpenAI-compatible) — record status: unverified — https://www.deai.org/tracker/engy
- io.net (GPU marketplace, tier 1), API live — record status: unverified — https://www.deai.org/tracker/ionet
- Morpheus (Standalone network, tier 1), API live (OpenAI-compatible) — record status: unverified — https://www.deai.org/tracker/morpheus
- NEAR AI (Standalone network, tier 2), API live — record status: unverified — https://www.deai.org/tracker/near-ai
- Nesa (Standalone network, tier 2), API live — record status: unverified — https://www.deai.org/tracker/nesa
- Oasis Network (TEE play, tier 2), API live — record status: unverified — https://www.deai.org/tracker/oasis
- OpenServ (Standalone network, tier 2), API live — record status: unverified — https://www.deai.org/tracker/openserv
- Phala Network (TEE play, tier 1), API live — record status: unverified — https://www.deai.org/tracker/phala
- SOLAI (GPU marketplace, tier 2), API live — record status: unverified — https://www.deai.org/tracker/solai
- SolRouter (TEE play, tier 2), API live — record status: unverified — https://www.deai.org/tracker/solrouter
- Targon (Bittensor subnet, tier 1), API live — record status: unverified — https://www.deai.org/tracker/targon
- Venice AI (Standalone network, tier 1), API live — record status: unverified — https://www.deai.org/tracker/venice
- Warden Protocol (GPU marketplace, tier 2), API live — record status: unverified — https://www.deai.org/tracker/warden