llama-cpp-python

Package — LLM/AIPython 3.7+Advanced

Python bindings for llama.cpp: run LLMs locally on CPU/GPU

Quick Info

Documentation
Official Docs
Python Version
3.7+
Dependencies
NumPy, diskcache, typing-extensions (C++ compilation required)
Install
pip install llama-cpp-python

Learn by Difficulty

Easy2 examples
Examples|Playground
Intermediate1 example
Examples|Playground
Advanced1 example
Examples|Playground
Expert1 example
Examples|Playground

Quick Example

python
# Install: pip install llama-cpp-python
import llama_cpp_python

# Basic llama-cpp-python usage
print(f"Using llama-cpp-python")
# See documentation for detailed examples

llama-cpp-python is a third-party package. Python bindings for llama.cpp: run LLMs locally on CPU/GPU. Install with: pip install llama-cpp-python

Try in Playground

Tags

packagellmaigenerative-aiagents