Mmastodon TechnologySoftware first seen 14 h ago, last 12 h ago, peak #4
Developer flags paying twice for identical LLM answers
Original: I kept paying for the same LLM answer twice. An agent retries, a user double-clicks, two workers ask... # rust # llm # a
A software developer describes repeatedly being charged for the same large language model answer, caused by duplicate requests: an agent retrying, a user double-clicking, or two workers sending the same query. They outline a fix using embeddings to detect and reuse identical prompts instead of paying for duplicate API calls again.
Why now: Developers are looking for ways to cut wasted LLM API costs from duplicate requests.
LLM APIsRustopen source developers
Evidence
- I kept paying for the same LLM answer twice. An agent retries, a user double-clicks, two workers ask... # rust # llm # ai # opensource # software # coding # development # engineering # inclusive # community I stopped paying twice for the same LLM answer (and why embeddings… · hackaday@www.urbanmind.net · 3
API: https://socialmediatrends-api.osmike.com/v1/trends/1663814