⬢github C · 332 ★ +297 since we first saw it · pushed 12 min ago · GPL-3.0
prabanta-dev/janas
37 tok/s from Qwen3-Next-80B (48 GB) on a 32 GB laptop, 24 without drafts vs 2.8 for llama.cpp; reading 62 vs 16 tok/s. Faster than llama.cpp on all 13 models compared in Q4_K_M. LLMs bigger than your RAM, pure C, CPU or GPU.
Janas is an experimental LLM inference engine written in pure C that runs models larger than your RAM by streaming mixture-of-experts weights from an NVMe SSD, using speculative decoding, SIMD, and CPU or Vulkan GPU. Benchmarks claim it outperforms llama.cpp on all 13 compared Q4_K_M models, e.g. 37 tok/s for a 48 GB model on a 32 GB laptop.
Why now: It just appeared on Hacker News with striking benchmark claims of beating llama.cpp on every tested model, including running models bigger than system memory, which is drawing attention and scrutiny.
Who it is for: Enthusiasts and developers running local LLMs on consumer laptops or desktops with limited RAM who want faster inference without big GPUs.
avx2ccpu-inferenceggufinference-enginelinuxllmlocal-llmmixture-of-expertsmoe
Stars over our 106 snapshots: 35 to 332, since 1 d ago.
Where people talked about it
API: https://socialmediatrends-api.osmike.com/v1/repos/prabanta-dev/janas