Yhn first seen 8 h ago, last 11 min ago, peak #21
Janas runs LLMs larger than RAM via SSD streaming
Original: Janas: Run LLMs larger than RAM by streaming experts from SSD
A developer has released Janas, an open-source tool that lets large language models bigger than a machine's RAM run on local hardware by streaming model experts from an SSD on demand. The project, hosted on GitHub, targets the MoG/MoE approach where only parts of the model need to be active at once, making it easier to run very large models on consumer machines.
Why now: Interest in running large open-source language models on modest local hardware keeps growing, and SSD-streaming promises a cheaper path to that.
Rank over time, top of the chart is #1. 36 snapshots from 8 h ago to 11 min ago.
Evidence
- Janas: Run LLMs larger than RAM by streaming experts from SSD · khalidkhair · 127
API: https://socialmediatrends-api.osmike.com/v1/trends/2042096