DeepSeek open-sources DeepInfer, the serving stack behind V4.1-Flash

DeepSeek open-sourced DeepInfer, the inference framework that runs its production V4.1-Flash traffic, under Apache-2.0 on October 20. X engagement hit roughly 10.2k likes on the announcement within hours, and the repo now shows 13.6k stars. Builders who serve DeepSeek weights just got the vendor's own serving recipe.

Key facts

• DeepInfer is Apache-2.0 licensed and supports DeepSeek-V4.1-Flash as its default checkpoint, per the README.
• Supported features include FlashMLA, DeepGEMM, and DeepEP kernels, speculative decoding with confidence-scheduled verification, batch-invariant and deterministic sampling, and structured outputs (README).
• Supported hardware is NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs; builds need CUDA 13 or CANN 9.2 respectively (README).
• A complete service needs a KV cache system, VE/Prefill/Decode disaggregation, and cross-replica load balancing, which are "not included in this release" (README).
• DeepInfer "was originally built primarily on vLLM, with additional inspiration from SGLang and TensorRT-LLM," per the repo's own acknowledgments.
• X velocity: the @deepseek_ai announcement drew a Grok point-in-time snapshot of ~10.2k likes and ~476k views within 2-3 hours on October 20, roughly 50x the rest of the day's beat; these are unverified platform figures.

What's driving the conversation

The announcement came from @deepseek_ai on October 20, and the technical thread that carried the most weight came from @tianyi, walking through the kernel stack and scheduler design. The vLLM project's quote post acknowledged the lineage nod, which mattered to the ecosystem audience: DeepInfer is built primarily on vLLM by DeepSeek's own acknowledgment, though the repo does not document what was upstreamed versus forked, so any claim about the vLLM ecosystem impact is speculation and we are not making one.

Secondary coverage followed within hours, including MarkTechPost, which noted the offline-inference-only scope that most hot takes skipped.

The substance

Everything load-bearing here checks against the repo itself, which we read directly. DeepInfer is real, public, Apache-2.0, and the feature list matches what the announcement thread claimed: optimized kernels from the FlashMLA/DeepGEMM/DeepEP family, DSpark speculative decoding with confidence-scheduled verification, batch-invariant inference and deterministic sampling on NVIDIA, and structured outputs constrained by regex, JSON schema, or EBNF grammar.

Two limits matter as much as the features. First, this is offline inference; the KV cache system, prefill/decode disaggregation, and prefill/decode load balancing that a complete service needs are explicitly not included, so nobody is running DeepSeek's actual production architecture out of the box. Second, the benchmark charts in the repo (4 SM100 GPUs for prefill, 64 for decode) are DeepSeek's own measurements. There is a reproduction script, run_sweep.sh, but no independent benchmark has reproduced them yet. Treat every throughput figure as self-reported until someone else runs it.

The 155T tokens per day production-scale figure that circulated in secondary coverage is DeepSeek's own claim about its served traffic, not a measurement anyone outside the company has verified. We repeat it here only as a claim.

Why builders are watching

The significance is access, not benchmarks. The serving recipe behind one of the most heavily used open-weight API endpoints is now inspectable and self-hostable, which narrows the information gap between "call the API" and "run it yourself." Inference marketplaces and self-hosters already serving V4.1-Flash weights on vLLM can read how the vendor itself schedules batches, handles speculative decoding, and pins determinism, then compare against their own setups. We tracked the model itself in the V4.1-Flash 1017 update, the vLLM relationship in Bittensor's subnet 10 upstream merge (our coverage), and the underlying cost math in self-hosting versus API cost.

What to watch: whether the missing service components (KV cache, disaggregation, load balancing) ship in a follow-up release, and whether any independent party reproduces the repo's throughput numbers on their own hardware.