Mmastodon TechnologyMobile first seen 9 h ago, last 9 h ago, peak #2
Why Phones Run LLMs Far Slower Than Expected
Original: Why Your Phone Runs LLMs 80x Slower Than It Should (A Debugging Log) Tags: on-device... # android # llm # performance #
A debugging write-up claims on-device large language models on Android phones can run up to 80 times slower than they should, and walks through the investigation to find the cause. The post covers performance issues in mobile LLM inference and what the developer found in the code, drawing interest from Android and AI developer communities.
Why now: Developers running AI models locally on mobile devices are interested in the technical causes of severe performance losses.
Evidence
- Why Your Phone Runs LLMs 80x Slower Than It Should (A Debugging Log) Tags: on-device... # android # llm # performance # kotlin # software # coding # development # engineering # inclusive # community Why Your Phone Runs LLMs 80x Slower Than It Should (And What I Found) · hackaday@www.urbanmind.net · 3
API: https://socialmediatrends-api.osmike.com/v1/trends/1613893