DeepInfer
DeepSeek's open-source production inference engine for V4.1-Flash, running on NVIDIA Blackwell GPUs and Huawei Ascend 950 NPUs.
Overview
DeepSeek open-sourced DeepInfer on October 20, 2026 under Apache 2.0, describing it as the inference framework behind its large-scale production services. It targets DeepSeek-V4.1-Flash and supports NVIDIA SM100 (Blackwell) GPUs and Huawei Ascend 950 NPUs. The README says it was built primarily on vLLM with ideas from SGLang and TensorRT-LLM, and bundles DeepSeek's FlashMLA, DeepGEMM, DeepEP, TileKernels and DeepSelect kernels. DeepSeek cites about 155 trillion tokens per day per cluster and a 96% input cache hit rate in production; these figures are self-reported and not independently tested.
This is a sourced directory entry. Features and capabilities are reported in the linked sources; this entry does not document independent Aymorphic testing.
Key features
- Apache 2.0 license; code at github.com/deepseek-ai/DeepInfer
- Targets DeepSeek-V4.1-Flash on NVIDIA SM100 and Huawei Ascend 950
- Attention data parallelism with balanced context parallelism; MoE expert parallelism with EPLB
- Draft-token decoding acceleration with confidence-scheduled verification (DSpark)
- Asynchronous zero-overhead batch scheduler and batch-invariant mode
- Constrained generation via regex, JSON schema or EBNF grammar
Use cases
- Self-hosting V4.1-Flash at high throughput on Blackwell or Ascend clusters
- Reference for MoE expert-parallel serving and speculative decoding
- Deterministic, reproducible sampling for evaluation and RL rollouts
Pricing
Free and open source (Apache 2.0); hardware costs are the user's own.
Pricing reflects the source record and may change. Confirm current terms with the provider.
Dates and sources
- Released
- 20 Oct 2026
- Record updated
- 28 Oct 2026
- Record last checked
- 28 Oct 2026