⬢github Python · 16.6K ★ +1.5K since we first saw it · pushed 1 h ago · Apache-2.0
FlashML-org/FreeToken
FreeToken brings datacenter-scale model serving to your desktop. Run massive models locally, fast and efficiently.
FreeToken is an edge-native inference engine for serving large open-weight Mixture-of-Experts models on consumer hardware like gaming PCs and laptops. It combines GPUs, CPUs, and host memory into one elastic serving platform, with expert caching, semantic-aware KV reuse, runtime VRAM reallocation, quantization support, and OpenAI/Anthropic-compatible APIs plus a desktop app and CLI.
Why now: Buzz around running frontier-scale MoE models (e.g., 290B+ DeepSeek-V4-Flash, Qwen, GLM) fast on consumer RTX GPUs, backed by a recent paper, a Windows/Linux desktop app, and integration with popular coding agents like Codex and Claude Code.
Who it is for: Developers and enthusiasts who want to run large open-weight MoE models locally on gaming PCs or workstations for chat, agents, and coding workflows.
deepseek-v4edgeglminferenceminimaxmoeqwentransformer
Stars over our 332 snapshots: 15.1K to 16.6K, since 3 d ago.
Where people talked about it
- ⬢github FlashML-org/FreeToken 1 d ago
API: https://socialmediatrends-api.osmike.com/v1/repos/FlashML-org/FreeToken