vLLM
Open-source LLM inference/serving engine; v0.31.0 (October 13, 2026) adds GA disaggregated prefill/decode serving and day-0 support for Grok 5 and Mistral Large 4.
Overview
vLLM shipped v0.28.0 (August 26, 2026), v0.29.0 (September 9, 2026), v0.30.0 (September 22, 2026), and v0.31.0 (October 13, 2026) in this window. v0.30.0 added 'Fast Start,' a persistent per-GPU daemon that holds pre-quantized, tensor-parallel-sharded weights in GPU memory and maps them via CUDA IPC instead of disk reads, plus Gumbel-max watermarking and DeepSeek-V4.1-Flash support. v0.31.0 moves disaggregated prefill/decode serving (splitting prompt-processing and token-generation across separate GPU pools) from experimental to generally available, cutting p99 time-to-first-token under mixed long/short-prompt load, and adds day-0 inference support for xAI's newly announced Grok 5 and Mistral Large 4.
This is a sourced directory entry. Features and capabilities are reported in the linked sources; this entry does not document independent Aymorphic testing.
Key features
- GA disaggregated prefill/decode serving across heterogeneous GPU pools
- Fast Start: persistent GPU daemon with CUDA IPC weight mapping for near-instant cold starts
- Gumbel-max watermarking for AI-generated text detection
- Day-0 serving support for Grok 5 and Mistral Large 4
- Regular release cadence with DeepSeek-V4 optimizations and tiered KV offloading
Use cases
- Self-hosted high-throughput LLM serving
- Fast-cold-start autoscaling deployments
- Mixed long-context/short-context production traffic needing disaggregated serving
- Open-weight and newly released model deployment at scale
Pricing
Free, open source (Apache 2.0).
Pricing reflects the source record and may change. Confirm current terms with the provider.
Dates and sources
- Released
- Not listed
- Record updated
- 14 Oct 2026
- Record last checked
- 14 Oct 2026