⬢github first seen 14 h ago, last 9 min ago, peak #7
AirLLM runs 70B models on a single 4GB GPU
Original: lyogavin/airllm
The open-source project AirLLM, published on GitHub by developer lyogavin, enables inference with large language models as big as 70 billion parameters on a single consumer GPU with just 4GB of memory. The repository, written largely as Jupyter Notebooks, has drawn attention for dramatically lowering the hardware barrier to running top-tier open models locally, a capability that normally requires tens of gigabytes of VRAM.
Why now: Running a 70B-parameter model on a 4GB GPU challenges assumptions about the hardware needed for local large language model inference.
Rank over time, top of the chart is #1. 57 snapshots from 14 h ago to 9 min ago.
Evidence
- lyogavin/airllm · lyogavin · 16
AirLLM 70B inference with single 4GB GPU
API: https://socialmediatrends-api.osmike.com/v1/trends/1850728