⬢github C++ · 4 ★ +4 since we first saw it · pushed 1 d ago · MIT
hardwiredagent/silicon-forge-arm-neon-study
Arm NEON kernel for llama.cpp tiled K-quant matmul - benchmark study, patch and tools
A small patch and benchmark study that adds a missing Arm NEON inner kernel to llama.cpp's tiled K-quant matrix multiplication. Formats like IQ3 and Q3_K previously fell back to a slow unpack-per-dot-product path on Arm; this 164-line change using SDOT makes prompt processing up to 6.5x faster on Neoverse server cores, with full benchmarks and tooling included.
Why now: It was discussed on Hacker News as 'llama.cpp: 6.5x faster prompt processing on Arm with a NEON kernel' — a notable new optimization benchmark for a popular project.
Who it is for: Developers running llama.cpp on Arm servers or devices who want faster prompt processing for low-bit quantized models.
Stars over our 159 snapshots: 0 to 4, since 1 d ago.
Where people talked about it
API: https://socialmediatrends-api.osmike.com/v1/repos/hardwiredagent/silicon-forge-arm-neon-study