⬢github Python · 2 ★ · pushed 7 d ago · 0BSD
me-farza/lightreason
token speedtup
Lightreason is a local inference toolkit and interactive Studio for running Qwen models faster on a single RTX 4090. It combines CUDA Graphs, speculative draft-model decoding, and CPU offloading of expert weights, reporting measured speedups (e.g. ~6x on Qwen3.6-35B) while keeping the original BF16 Qwen model as the final authority on tokens.
Why now: A recent Hacker News post claims a 6-11x speedup running Qwen3.6 on an RTX 4090 with the full repo shared, drawing attention to its benchmark evidence.
Who it is for: Developers and hobbyists with a consumer GPU who want to run Qwen models locally at higher token speeds.
Stars over our 836 snapshots: 2 to 2, since 8 d ago.
Where people talked about it
API: https://socialmediatrends-api.osmike.com/v1/repos/me-farza/lightreason