Mmastodon TechnologyAI first seen 2 d ago, last 12 h ago, peak #11
Tsinghua researchers open-source AKV cache system for LLM agents
Original: 📰 Tsinghua AI Open-Sources AKV, a Cache Engineering Server for LLM Agents on SGLang Tsinghua researchers released AKV, a
Researchers at Tsinghua University have released AKV, an open-source serving system built on SGLang that applies cache engineering to large language model agents. Instead of summarizing conversation history, the system drops and repositions KV cache entries, reportedly cutting average serving costs by 27.7%. The release is drawing attention from developers tracking cheaper ways to run long-running AI agents.
Why now: Open-source tooling that meaningfully lowers the cost of serving LLM agents is of immediate interest to the AI developer community.
Rank over time, top of the chart is #1. 2 snapshots from 2 d ago to 12 h ago.
Evidence
- 📰 Tsinghua AI Open-Sources AKV, a Cache Engineering Server for LLM Agents on SGLang Tsinghua researchers released AKV, an SGLang-based serving system for cache engineering. LLM agents drop and reposition KV cache instead of summarizing history, cutting average cost 27.7% across… · feed@igeek.gamer-geek-news.com
- 📰 Tsinghua AI Open-Sources AKV, a Cache Engineering Server for LLM Agents on SGLang Tsinghua researchers released AKV, an SGLang-based serving system for cache engineering. LLM agents drop and reposition KV cache instead of summarizing history, cutting average cost 27.7% across… · feed@igeek.gamer-geek-news.com
API: https://socialmediatrends-api.osmike.com/v1/trends/3719936