✦
Summary
DeepInfer is DeepSeek's inference framework for efficient, large-scale execution of supported DeepSeek models. It combines optimized kernels, expert parallelism, asynchronous scheduling, and accelerated verification to support offline inference and serving-system development.
Key Features
DeepSeek-V4.1-Flash support.
Attention and expert parallelism.
Expert load balancing.
Asynchronous batch scheduling.
Batch-invariant inference.
Deterministic sampling.
Regex and schema constraints.
NVIDIA and Ascend hardware paths.
About DeepInfer
Infrastructure engineers can build around the released runtime while supplying the additional caching and load-balancing components required for a full service. Core capabilities include DeepSeek-V4.1-Flash support, Attention and expert parallelism, Expert load balancing, and Asynchronous batch scheduling.
Public code or model weights are linked from the project, with setup instructions for running or adapting the system. Additional characteristics include Batch-invariant inference, Deterministic sampling, Regex and schema constraints, and NVIDIA and Ascend hardware paths.