⬢github Python · 2.4K ★ +157 since we first saw it · pushed 49 min ago · MIT
lightseekorg/tokenspeed
TokenSpeed is a speed-of-light LLM inference engine.
TokenSpeed is an open-source LLM inference engine built for agentic workloads, aiming for TensorRT-LLM-level performance with vLLM-level usability. It separates a C++ control plane (a finite-state machine enforcing KV cache safety at compile time) from a Python execution plane, and uses a pluggable kernel system with fast MLA attention on Blackwell GPUs, backed by the non-profit LightSeek Foundation.
Why now: Recent high-profile momentum: it joined the PyTorch Ecosystem, and 2026 posts showcase record throughput (685 tokens/s per user on GB300) plus Day-0 support for frontier models like Kimi K3, Qwen3.8, and GLM 5.3 Flash on NVIDIA and AMD hardware.
Who it is for: Teams running production LLM serving on high-end NVIDIA or AMD accelerators, especially for agentic applications.
b200b300blackwelldeepseekgb200gb300glmgpt-ossinklingkimi
Stars over our 34 snapshots: 2.3K to 2.4K, since 9 h ago.
Where people talked about it
- ⬢github lightseekorg/tokenspeed 2 min ago
API: https://socialmediatrends-api.osmike.com/v1/repos/lightseekorg/tokenspeed