⬢github first seen 2 d ago, last just now, peak #7
AirLLM runs 2.8T parameter model on single 4GB GPU
Original: lyogavin/airllm
The open-source AirLLM project by developer lyogavin claims to enable inference of 2.8-trillion-parameter large language models on a consumer GPU with as little as 4GB of memory. The tool, written in Python and packaged for Jupyter notebooks, is drawing attention from developers interested in running huge models without expensive hardware. Enthusiasts are debating its practicality, since memory-saving techniques often trade speed for feasibility.
Why now: It offers a strikingly cheap way to run enormous AI models on ordinary hardware, a hot topic among developers.
Rank over time, top of the chart is #1. 24 snapshots from 5 h ago to just now.
Evidence
- lyogavin/airllm · lyogavin · 16
AirLLM 2.8T LLM inference with single 4GB GPU
API: https://socialmediatrends-api.osmike.com/v1/trends/1850728