A high-throughput and memory-efficient inference and serving engine for LLMs
其他人工智能更新于 2026-10-25
其他人工智能Apache-2.0
Repository dossier
vllm-project/vllm-omni
A framework for efficient model inference with omni-modality models
Brief / README 摘要
查看原文Easy, fast, and cheap omni-modality model serving for everyone [2026/10] We released the vLLM-Omni Technical Report, describing a unified serving runtime for text, audio, image, video, and action generation. [2026/09] We released 0.30.0, rebased onto vLLM 0.30.0, featuring a unified full-duplex serving framework around engine-owned sessions for MiniCPM-o 4.5 and AURA, native cross-stage KV and multimodal payload transfer (Mooncake AR-to-DiT handoff and NIXL connectors), interactive world-model serving with LingBot World, and realtime MiniMax H3 Turbo inference on NVIDIA Blackwell and Ascend 95…
Topics / 标签
01
浏览全部相关推荐
🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
其他人工智能更新于 2026-10-25
The most powerful and modular diffusion model GUI, api and backend with a graph/nodes interface. The fastest local inference engine in the world.
其他人工智能更新于 2026-10-25
Implement a ChatGPT-like LLM in PyTorch from scratch, step by step
其他人工智能更新于 2026-10-02