⬢github C++ · 312 ★ +264 since we first saw it · pushed 3 d ago · Apache-2.0
huawei-csl/MACaQ
SOTA model-agnostic quantization for GGUF, local inference and beyond. Quantize and run text, image, video and audio models with llama.cpp, vLLM, Diffusers and PyTorch.
MACaQ is Huawei's open-source, model-agnostic quantization toolkit that produces high-quality GGUF and safetensors models for local inference across text, image, video, and audio. It targets llama.cpp, vLLM, Diffusers, and PyTorch backends, supports 2–4 bit formats, and claims fast, memory-efficient calibration with quality better than existing methods like MR-GPTQ and DuQuant++.
Why now: Huawei just released its previously internal quantizer alongside an arXiv paper, drawing Hacker News attention because its speed and quality claims beat current state-of-the-art GGUF quantization tools.
Who it is for: Developers and researchers running or serving large generative models locally on consumer or Ascend hardware.
ascendaudioaudio-generationdiffusersggufgguf-quantizationimage-generationllama-cppllmlocal-ai
Stars over our 401 snapshots: 48 to 312, since 4 d ago.
Where people talked about it
API: https://socialmediatrends-api.osmike.com/v1/repos/huawei-csl/MACaQ