AI benchmarks

A new open-source project called Strata claims to run the Qwen 3.8 Flash Next model, a 125-billion-parameter language model, on consumer hardware like an Nvidia RTX 4090, reportedly reaching around 100 tokens per second. The claim has drawn attention from developers discussing whether such performance on a single consumer GPU is realistic and what it could mean for local AI inference.

OpenAI has published a new post outlining how its AI systems are advancing on mathematical reasoning and problem-solving, and how it evaluates and shares that progress. The announcement is drawing attention on Hacker News, where it is among the most engaged items, with readers debating how significant the mathematical capabilities are and how honestly such progress is being reported.

A new project called Jevman has been launched, using the classic arcade game Pac-Man as a benchmark environment for testing AI decision-making models. Announced on Hacker News, the tool from Opper AI invites developers and researchers to see how well language and decision models navigate the game's maze, weighing trade-offs and planning under uncertainty. Early commenters are discussing the approach and what game-based benchmarks reveal about reasoning ability.

Anthropic has restricted public internet access to its internal evaluations, according to a report by The Verge. The move means the AI company's benchmark testing and safety assessments will no longer be reachable online, drawing attention as AI firms face scrutiny over transparency of how they measure model safety and performance.

A new study by Epoch AI finds that recent AI models struggled to match a human algorithmic innovation when tested on reproducing novel research ideas. The findings suggest frontier models still lag behind humans on tasks requiring genuine creative problem-solving, even as they excel at routine coding and reasoning benchmarks.