deepseek-ai/DeepInfer

DeepInfer is the high-performance inference framework that powers DeepSeek's large-scale production services. This open-source release provides offline inference with the core features needed for serving at scale. Optimized kernels, model and sampler runtime Parallelism: Attention DP / balanced CP, MoE EP + EPLB DSpark and confidence-scheduled verification SWA bounded replay Asynchronous scheduling (zero-overhead batch scheduler) Batch-invariant inference Deterministic sampling Structured outputs Feature details and configuration Optimized kernels: FlashMLA, DeepGEMM, DeepEP, TileKernels, Deep…