search
open-source AI agent
Trends
- 1Nvidia Unveils Open-Source Security System for Rogue AI Agents▼Nvidia’s Answer to Rogue Agents Is an Open-Source AI Security System
Nvidia has introduced an open-source security system designed to guard against rogue AI agents, according to WIRED. The system aims to address growing concerns that autonomous AI tools could act unpredictably or outside their intended instructions. The move positions Nvidia within an emerging effort to build safety tooling for AI agents, and the choice to open-source it is drawing attention as companies weigh transparency against control in AI security.
- 2Nvidia launches open-source platform to improve AI agent safety▼Nvidia launches open-source platform to enhance AI agent safety
Nvidia has introduced an open-source platform designed to make AI agents safer to deploy, giving developers tools to test and guard against unsafe behaviour. The move, reported by SC Media, positions Nvidia alongside other major tech firms racing to address security and reliability concerns as autonomous AI systems spread across enterprise software.
- 3Nvidia Launches Open-Source Platform to Stop Rogue AI Agents▼Nvidia Launches Open-Source Platform Aimed at Keeping AI Agents From Hacking Other Sites
Nvidia has released an open-source platform designed to prevent AI agents from hacking or compromising other websites and systems. The tooling aims to add safety guardrails to autonomous agents as they browse and interact with the web. The move positions Nvidia to shape security standards for the fast-growing agentic AI field, and it is likely to draw attention from developers and security researchers assessing how well it works.
- 4
NVIDIA has released its AI agent safety platform as open source, according to technology news coverage. The move makes the tooling for monitoring and safeguarding autonomous AI agents publicly available, allowing developers to inspect, adapt and build on the software rather than rely on a proprietary offering. Coverage so far has stuck to the announcement, with no detailed reactions from the developer community yet.
- 5Nvidia Releases Open-Source AI Security System▼Nvidia’s Answer to Rogue Agents Is an Open-Source AI Security System https:// fed.brid.gy/r/https://www.wire d.com/story
Nvidia has unveiled an open-source security system designed to protect against rogue AI agents, according to Wired. The tool aims to address risks from autonomous AI systems acting beyond their intended limits, and its open-source release is drawing attention from developers and security researchers weighing how the industry should police increasingly capable agents.
- 6Nvidia unveils open-source system to keep AI agents in check▼Nvidia unveils open-source security system to stop AI agents going out of human control
Nvidia has announced an open-source security system designed to prevent AI agents from acting beyond human control. The tool is aimed at keeping autonomous AI systems within defined safety boundaries as companies increasingly deploy agents that act with minimal supervision. The announcement is drawing attention across the tech industry, where concerns about AI alignment and oversight are growing alongside rapid adoption of agentic systems.
- 7AI agents used to steal 600,000 credit card records●Security company Gambit says an attacker used three open-source # AI agents to breach 27 companies and steal more than 6
Security company Gambit reports that an attacker used three open-source AI agents to breach 27 companies and steal more than 600,000 credit card records. According to the firm, each automated scan cost roughly $25, making the operation cheap and highly scalable. The case is raising alarm about how easily accessible AI tools can be repurposed for large-scale cybercrime against financial targets.
- 8GitHub's AI agent finds 24 Android vulnerabilities●GitHub found 24 Android vulnerabilities using its open-source AI security agent: automated vulnerability discovery on re
GitHub says it discovered 24 vulnerabilities in Android using its open-source AI security agent, describing it as automated vulnerability discovery at scale on real production software. The company published details in a blog post, and the report is drawing attention as a notable demonstration of AI agents doing practical security research on widely used code.
- 9
Hindsight is an open-source Python project from vectorize-io described as 'Agent Memory That Learns'. It is aimed at developers building AI agents, giving them a memory system that improves over time rather than storing static context. Evidence is limited to the repository itself, so specific user reactions or discussion themes are not visible in the posts. Its appearance high on the trending list suggests strong recent attention from the developer community, though the exact trigger is unclear from the available evidence.
- 10Google Research Open-Sources RRSI Self-Improving AI Agents▼Google Research Open-Sources RRSI: AI Agents That Improve Their Own Harness Without Overfitting
Google Research has open-sourced RRSI, a framework allowing AI agents to refine their own evaluation harness while guarding against overfitting. Announced via MarkTechPost, the release lets developers inspect and build on the underlying code. The announcement is drawing attention from AI practitioners interested in agent self-improvement methods that remain reliable rather than gaming their own benchmarks.
- 11Nvidia Launches Open-Source Platform to Stop Rogue AI Agents▼Nvidia Debuts Open-Source Platform to Contain Rogue AI Agents in Milliseconds
Nvidia has introduced an open-source platform designed to contain misbehaving AI agents within milliseconds. The tool is aimed at developers building autonomous AI systems, allowing them to cut off agents that act outside their intended boundaries before damage is done. It marks a step by Nvidia into AI safety tooling alongside its dominant AI hardware business.
- 12Hackers Weaponize Open-Source AI Agent Against Docker Servers▼Hackers Turn an Open-Source AI Agent Into a Tool for Controlling Compromised Docker Servers
Hackers have repurposed an open-source AI agent into a command-and-control tool for operating compromised Docker servers. Security researchers report attackers are using the agent's automation capabilities to run commands, move laterally and manage infected containers remotely. The case highlights a growing trend of malicious actors abusing legitimate AI tooling, and Docker deployments left exposed online remain a prime target for takeover.
- 13
Univer is an open-source TypeScript project from dream-num that bills itself as an 'Office Harness for AI Agents'. It provides a single runtime combining spreadsheets, documents, slides, canvas, relational tables, and PDF handling. The repository is trending on GitHub, and the framing suggests developers are interested in giving AI agents tools to create and manipulate office-style documents. Beyond the project's own description, there is little discussion in the available evidence explaining what users are saying about it.
- 14Stanford and Nvidia release CLM-8B agent model▼Stanford and Nvidia's open CLM-8B caches reusable agent actions and runs up to 9x faster than Jev in tests
Stanford University and Nvidia have open-sourced CLM-8B, an AI model built for software agents that caches reusable actions instead of recomputing them. In tests the model ran up to nine times faster than Jev, a comparable agent system. The open release is drawing attention for offering large speed gains on agentic workloads, an area where inference cost is a major bottleneck for developers.
- 15Cloudflare launches free agentic threat intelligence tooling▼Introducing Threat Signals: agentic skills for open-source threat intelligence, free for every Cloudflare account
Cloudflare has introduced Threat Signals, a set of agentic skills for open-source threat intelligence that the company says will be free for every Cloudflare account. The announcement, made on the Cloudflare blog, positions the tooling as a way for customers of all sizes to tap into threat intelligence capabilities without additional cost. Reaction so far has focused on the free availability and the use of agentic AI in security operations.
- 16SGLang turns Qwen model into Pokémon-playing engine●SGLang Transforms Qwen Model into Pokémon-Beating Decision Engine
Engineers have adapted the Qwen language model, served through the SGLang framework, into a decision engine capable of beating Pokémon battles. Reports describe the model evaluating game states and selecting winning moves in real time. The project is drawing attention in AI circles as an example of open-weight models being pushed beyond chat tasks into competitive game-playing agents.
- 17Corral tool kills every command an AI agent starts▼Show HN: Corral – Kill every command your agent starts
A new open-source tool called Corral has been released on GitHub, letting developers automatically kill every command that an AI coding agent starts. The tool gives users a single way to terminate processes spawned by agents, addressing concerns about runaway commands or unwanted background tasks during automated coding sessions.
- 18MLC releases TIRx, an open compiler harness for agentic GPU programming●TIRx Harness: An Open Compiler Harness for Agentic GPU Programming
The MLC team has introduced TIRx Harness, an open-source compiler harness designed for agentic GPU programming, where AI agents write and optimise GPU code. The announcement was shared by a member of the team and is drawing attention from developers interested in compiler tooling and AI-driven systems programming.
- 19Keeping up with AI open source is a full-time job●Keeping up with AI open source is a full-time job. Every week a new agent framework, MCP server or... # ai # opensource
Developers are commenting on the overwhelming pace of open-source AI releases, with new agent frameworks, MCP servers and machine learning tools appearing every week. Discussion centres on which fast-growing projects are worth attention and how hard it has become for engineers and hobbyists to stay current in the field.
- 20
The OpenClaw Foundation has announced the launch of an open-source platform for building AI agents. The release gives developers a freely available foundation for creating autonomous software agents, adding to the growing number of open-source alternatives in the AI tooling space. Details on features, licensing and backing have not yet been widely reported.
- 21Corral kills every command your AI agent starts●Show HN: Corral kill every command your agent starts
A developer has released Corral, an open-source tool that automatically kills every command started by an AI coding agent, giving users a way to rein in runaway processes. The project, shared on GitHub, is being discussed on Hacker News as interest grows in safety controls for autonomous agents.
- 22OpenAI, Red Hat and NVIDIA Back Open-Source Agent Control Plane●OpenAI, Red Hat, and NVIDIA Back an Open-Source Agent Control Plane — While OpenAI Ships a Proprietary One
OpenAI, Red Hat and NVIDIA are jointly backing an open-source control plane for managing AI agents, even as OpenAI simultaneously releases its own proprietary version of the technology. The initiative would give developers a shared, vendor-neutral way to orchestrate and govern autonomous agents, while the parallel commercial offering suggests the companies see both collaboration and competition in the fast-growing agent infrastructure space.
- 23
Tiledesk has raised €1.1 million to develop its open-source platform for building AI agents. The funding will support the company's work on conversational and AI-powered customer engagement tools. The news is circulating in software and startup circles, where interest in open-source alternatives to proprietary AI agent platforms is growing.
- 24IQuest Research Open-Sources 320B Agentic Coding Model●IQuest Research Open-Sources IQuest-Q1, a 320B MoE Model for Agentic Coding With 15B Active Parameters
AI startup IQuest Research has released IQuest-Q1, an open-source mixture-of-experts model with 320 billion total parameters but only 15 billion active per query, aimed at agentic coding tasks. The sparse architecture promises large-model capability with much lower inference costs. It arrives as competition intensifies among open-weight coding models, and developers are weighing its benchmarks and licensing against rivals like DeepSeek and Qwen.
Repos
- feder-cr/dots Open-source dots for the web: an AI agent with its own browser, one that does not get blocked.
- heyjunpenn/awesome-jev A verified, community-maintained catalog of 962 open-source projects built with Jev.
- LockedinLabs-AI/agent-console Local-first observability for AI coding agents. Every Claude Code and Codex session's tokens, cache, models and cos
- CopilotKit/openmuse A personal agent with a browser, terminal, files, and work that keeps going built with CopilotKit and AG-UI.
- debpalash/VoiceStudio VoiceStudio is the open-source, fully-local ElevenLabs alternative — voice cloning, voice design, video dubbing, dictati
- devdotfast/whiteboard open-source canvas for thoughtful software design
- paperclipai/paperclip The open-source app everyone uses to manage agents at work
- dream-num/univer The Office Harness for AI Agents — Spreadsheets, Docs, Slides, Canvas, Relational Tables, and PDF in one runtime.
- rohitg00/ai-engineering-from-scratch Learn it. Build it. Ship it for others.
- naw103/foremerge Catch intent conflicts before code conflicts. The open-source coordination protocol for coding agents, built above Git.
- SupercmoHQ/superCMO-skills Open-source skills that empower any AI agent (Claude, Cursor, Hermes, etc.) to generate end-to-end marketing campaigns -