Builders spent November 17 arguing about one line in Vercel's changelog: AI Gateway now accepts your own model endpoints as private providers. The mechanism is verified, the changelog is live, and the X post announcing it drew high velocity, roughly 188 likes and 24,000 views within a day.
Key facts
- Vercel's changelog documents Private Inference: team-owned endpoints callable as
private/<slug>IDs through Chat Completions, Responses or Messages APIs, translated to your endpoint's format. - Credentials are stored encrypted and models stay scoped to the owning team, absent from the public catalog.
- Zero Data Retention is an optional per-provider toggle, a configuration choice rather than a verified guarantee.
- The Vercel CLI can provision providers from version 65.16.0.
- The announcement post on X reached high velocity; a setup walkthrough from @buildlistxyz circulated alongside it the same day.
What's driving the conversation
Vercel's own announcement post started the thread, and the replies split into two camps. One camp reads this as validation for keeping open-weight self-hosting in the stack: register your fine-tune or your vLLM box as a private provider, keep catalog models for everything else, and get a single log surface across both. The buildlist walkthrough demonstrates exactly that flow with a few CLI lines, after which private/support-fine-tune behaves like any catalog model in the AI SDK.
The other camp is blunter about what this is and is not. A gateway in front of your endpoint does not make the endpoint managed: you still own capacity, patching and uptime, and if your box falls over at 3am, Vercel's optional catalog fallback routes around it with a closed model. That is convenience, not control. Several replies also noted the ZDR toggle is exactly the kind of claim DeAI's tracker treats as a policy statement until someone observes actual retention behavior.
The substance
What is verified: the feature exists, the changelog documents it, the CLI syntax works as shown, and private requests appear in the same logs, trace drains and reporting as catalog traffic. Per-provider Zero Data Retention is a real configuration option in the product. As a claim about data handling, though, it stays a claim until observed or attested, the same standard we applied to Cohere's Model Vault zero-retention docs.
What is commentary: most of the "this kills self-hosting tooling" and "this is the end of vendor lock-in" takes on X. Neither follows. The feature is a managed-gateway on-ramp for teams that already self-host; it neither removes the operational burden of running your endpoint nor locks you in more than any gateway does.
Why builders are watching
The relevant comparison is not Vercel versus self-hosting; it is the gateway layer consolidating. Register your own endpoint for the sensitive or custom workload, fall back to catalog models for the rest, and one audit surface covers both. For teams already running open-weight models on their own hardware, this lowers the integration cost of mixing the two; for teams evaluating whether self-hosting is worth the ops load, it adds another middle path. The compilation and inference tooling wave keeps reshaping what "self-hosted" costs, and managed gateways keep absorbing the edges of it. Watch whether other gateways copy the private-provider pattern within the quarter.
Questions
- What is Vercel AI Gateway Private Inference?
- A feature that lets a team register its own model endpoints, fine-tuned, self-hosted or prerelease, as private providers on Vercel AI Gateway. They are callable as private/<slug> IDs through the same authentication, SDKs and observability as catalog models, visible only to the owning team.
- Does Private Inference include Zero Data Retention?
- Zero Data Retention is available as an optional per-provider setting on registered private providers. It is a configuration toggle described in Vercel's changelog and docs, not an independently verified guarantee; whether retention behavior matches the toggle in practice has to be checked in traffic logs.
- Who is responsible if a private endpoint goes down?
- The endpoint operator. Vercel provides the gateway, auth, logging and optional fallback to a catalog model, but uptime and health of the self-hosted or fine-tuned endpoint remain the operator's responsibility.
Sources
About DeAI
DeAI is an independent publication covering open-weight AI models, private inference, and decentralized infrastructure — the tools for running AI you actually control. We test providers on price, privacy, and refusal behavior and publish the numbers, not the vibes. DeAI is powered by Morpheus (mor.org), a decentralized inference marketplace, and covers it on the same terms as every other provider.
Powered by Morpheus and StrandCMS
Morpheus is a decentralized inference marketplace, covered on the same terms as every other provider — we rank it wherever the data lands. StrandCMS is the open-source, agent-first framework this site is built on.
