vllm
Package — LLM/AIPython 3.7+Advanced
High-throughput LLM serving engine with PagedAttention
Quick Info
- Documentation
- Official Docs
- Python Version
- 3.7+
- Dependencies
- PyTorch, transformers, ray, sentencepiece, numpy, xformers (NVIDIA GPU required)
- Install
pip install vllm
Learn by Difficulty
Quick Example
python
# Install: pip install vllm import vllm # Basic vllm usage print(f"Using vllm") # See documentation for detailed examples
vllm is a third-party package. High-throughput LLM serving engine with PagedAttention. Install with: pip install vllm
Try in PlaygroundTags
packagellmaigenerative-aiagents