vLLM
High-throughput and memory-efficient inference and serving engine for Large Language Models. Deploy AI faster with state-of-the-art performance.
96.1K
GitHub Stars
More like vLLM
Tags
- inference-engine
- serving
- throughput
- open-source
High-throughput and memory-efficient inference and serving engine for Large Language Models. Deploy AI faster with state-of-the-art performance.