⬢github first seen 1 d ago, last 14 min ago, peak #7
AirLLM runs 70B model on a 4GB GPU
Original: lyogavin/airllm
A new open-source tool called AirLLM claims to enable inference of large 70-billion-parameter language models on consumer graphics cards with as little as 4GB of memory, far below what such models normally require. The project is written in Python with Jupyter notebooks and is attracting attention from developers interested in running big models cheaply on local hardware.
Why now: Running a 70B model on a single 4GB GPU would dramatically lower the hardware barrier for local large language model inference, which excites the AI developer community.
Rank over time, top of the chart is #1. 2 snapshots from 29 min ago to 14 min ago.
Evidence
- lyogavin/airllm · lyogavin · 16
AirLLM 70B inference with single 4GB GPU
API: https://socialmediatrends-api.osmike.com/v1/trends/1850728