NVIDIA-backed optimization work on vLLM significantly increased the inference performance of Moonshot AI’s Kimi K3 on GB300 GPUs.

In a controlled test using 16 NVIDIA GB300 GPUs and DSpark speculative decoding, Kimi K3 reached 842 decode tokens per second, about 1.73 times the throughput of the standard vLLM configuration. Other tested configurations delivered 1.5–2 times higher decode throughput and up to 1.8 times higher P90 interactivity on the AgentX coding-agent workload.

Source: vllm.ai