⬢github Jupyter Notebook · 37.4K ★ +1.3K since we first saw it · pushed 14 h ago · Apache-2.0
lyogavin/airllm
AirLLM 70B inference with single 4GB GPU
AirLLM is a Python library that dramatically shrinks inference memory requirements so very large open-source LLMs (70B up to 2.8T parameters) can run on small GPUs — e.g. a 70B model on a single 4GB card — without quantization, distillation, or pruning, by streaming model layers. It also adds training support for huge models on limited VRAM via streamed frozen weights plus adapters.
Why now: Recent updates announced support for brand-new flagship models like Kimi K3 (2.8T) and Qwen3.8-Flash-Next (125B) with striking low-VRAM numbers, plus a v3.0 with FP8 support and training on small GPUs.
Who it is for: Developers and researchers who want to run or fine-tune very large open-source LLMs locally on consumer GPUs with limited VRAM.
chinese-llmchinese-nlpfinetunegenerative-aiinstruct-gptinstruction-setllamallmloraopen-models
Stars over our 131 snapshots: 36.1K to 37.4K, since 1 d ago.
Where people talked about it
- ⬢github lyogavin/airllm 11 min ago
API: https://socialmediatrends-api.osmike.com/v1/repos/lyogavin/airllm