⬢github first seen 8 d ago, last 5 d ago, peak #7

AirLLM runs 2.8 trillion parameter models on a 4GB GPU

Original: lyogavin/airllm

The open-source project AirLLM, published on GitHub by developer lyogavin, enables inference on large language models with up to 2.8 trillion parameters using a single consumer GPU with just 4GB of memory. The tool, written largely as Jupyter Notebooks, is drawing attention from developers interested in running frontier-scale models without expensive data-center hardware.

Why now: Running trillion-parameter models on modest consumer hardware challenges assumptions about the cost of large-model inference.

AirLLMlyogavinGitHub

Open on github →

Evidence

• lyogavin/airllm · lyogavin · 16

  AirLLM 2.8T LLM inference with single 4GB GPU

API: https://socialmediatrends-api.osmike.com/v1/trends/1850728