⬢github Python · 114 ★ +26 since we first saw it · pushed 4 min ago · Apache-2.0
vllm-project/vllm-torchtpu
vLLM TorchTPU is a PyTorch-native TPU backend for vLLM, packaged as a plugin that runs vLLM's standard PyTorch model definitions on Google Cloud TPUs via torch.compile and custom Pallas kernels (attention, MoE, quantized matmul). It keeps the same vLLM CLI and OpenAI-compatible server, supports multi-host scaling, and works best on TPU v7x (Ironwood).
Why now: Newly announced (discussed on Hacker News as a PyTorch-native TPU back end for vLLM) and actively developed, repositioning the older JAX-based tpu-inference backend as vLLM JAX.
Who it is for: Teams serving LLMs with vLLM who want to run PyTorch model definitions on Google Cloud TPUs without changing model code.
Stars over our 259 snapshots: 88 to 114, since 4 d ago.
Where people talked about it
API: https://socialmediatrends-api.osmike.com/v1/repos/vllm-project/vllm-torchtpu