vLLM introduces day-one inference support for Kimi K3.1 Pro
vLLM Delivers Day-0 Support for Kimi K3.1 ProNov 4, 2026·22 min readvLLM supports Kimi K3.1 Pro on day 0: sparse attention on a KDA hybrid, a packed KV cache with FP8 or NVFP4 latents and MXFP4 index keys, token-sharded prefill, decode context parallelism, and fused low-latency decode.
vLLM announced day-one inference support for Moonshot AI's open-source Kimi K3.1 Pro model, fully adapting to the DSA sparse attention mechanism in its hybrid architecture. The system integrates MLA latent variables, index keys, and KDA states into a single block-level KV cache, supports NVFP4 and MXFP4 quantization, and achieves up to a 1.69x improvement in time-to-first-token latency on an 8-GPU GB300 system.
The post details the VRAM packing strategy for sparse attention and hybrid-state KV caching, providing an engineering reference for low-latency deployments of long-context, ultra-large MoE models.
Source:vLLM 官方博客（网页） · vllm.ai