⬢github Python · 942 ★ +851 since we first saw it · pushed 1 h ago · Apache-2.0
synchopate/llama-modchip
Run 284B DeepSeek-V4-Flash (<59GiB RAM: 41.1 cold pp, 9.1 N=1 tg, up to 25.3 spec tok/s) & 30B MoEs (22.4 N=1 / 111.6 spec tok/s) on 64GB CPU (0 GPUs).
llama-modchip is a Python tool for running large Mixture-of-Experts language models on CPUs alone, no GPU required. It fits a 284B DeepSeek-V4-Flash model into under 59 GiB of RAM (via a SwiGLU-H2 compression technique) at 256K context, runs 30B MoE models at 25–111 tok/s, uses Stockfish-based speculative drafting for speedups, and offers a 13ms Claude Code session restore.
Why now: It was featured on Hacker News for the striking claim of running a 284B-parameter model on a plain 64GB-RAM machine with zero GPUs, plus a fresh v2.4.0 release and a benchmark leaderboard.
Who it is for: Developers and hobbyists with CPU-only machines (or modest GPUs) who want to run frontier-scale and 30B-class MoE models locally.
ai-agentsapple-siliconavx512claude-codecpu-inferencedeepseekdeepseek-v4ggufhpcllama-cpp
Stars over our 347 snapshots: 91 to 942, since 3 d ago.
Where people talked about it
API: https://socialmediatrends-api.osmike.com/v1/repos/synchopate/llama-modchip