AI chatbots can say stupid, offensive, or dangerously inaccurate things. AI agents, however, bring a far more troubling possibility to the table: They can actually do stupid and dangerous things.
Ant Group believes it has a solution. The company’s AI Security Lab has released SingGuard-NSFA, an open-source security framework designed to inspect what autonomous AI agents are trying to do before they get permission to do it.
That distinction matters. A traditional chatbot generally waits for a question and produces an answer. An AI agent may browse websites, access files, call external tools, execute code, interact with business systems, or perform a chain of actions without stopping for approval after every step.
Giving an AI model those abilities can make it far more useful. It also creates a much larger security problem.
A malicious instruction hidden inside a webpage, document, email, or tool response could potentially manipulate an agent through prompt injection. Instead of simply returning a strange answer, the compromised agent might expose private information, misuse credentials, alter files, or run an unauthorized command.
SingGuard-NSFA is supposed to serve as a security checkpoint between the agent and the systems it can control. The framework examines requests and responses, looking for dangerous behavior before an autonomous action gets executed.
Ant Group says its framework covers 185 threat scenarios divided into seven categories. These include risks such as prompt injection, malicious code execution, privilege abuse, goal hijacking, and the misuse of legitimate tools.
Those are not hypothetical categories invented to market SingGuard-NSFA. The OWASP Top 10 for Agentic Applications identifies agent goal hijacking, tool misuse, identity and privilege abuse, unexpected code execution, and rogue agents among the most serious risks facing autonomous AI systems.
The framework includes a benchmark built from nearly 100,000 test samples across 133 languages. Ant Group is releasing several models through its inclusionAI organization, with versions ranging from 0.8 billion to 9 billion parameters.
According to the company, the smallest 0.8B model can compete with rival models containing roughly 8 billion parameters. Ant Group also claims the 9B version can detect threats with latency of approximately 50 milliseconds, potentially making it fast enough for real-time production use.
Those numbers sound impressive, but they should be treated as company claims for now. The SingGuard-NSFA research paper is not yet available, and independent researchers have not had much time to test the models. A security system can perform extremely well against a known benchmark while still missing attacks that use unfamiliar wording or techniques.
That is particularly important with prompt injection. Attackers do not need to use the same obvious phrases found in a training dataset. They can disguise malicious instructions, split them across different sources, encode them, translate them, or hide them inside content that otherwise appears harmless.
There is also a more fundamental limitation. An AI guardrail is still an AI model being asked to recognize when another AI model is about to behave dangerously. Adding a second model may reduce risk, but it does not automatically create a dependable security boundary.
Companies deploying agents should still limit permissions, isolate sensitive systems, validate tool inputs, log activity, and require human approval before high-risk actions. A guardrail should be one layer of protection, not an excuse to give an autonomous agent unrestricted access to everything.
SingGuard-NSFA nevertheless looks like a useful open-source contribution. The models and code are publicly available through GitHub and Hugging Face, allowing developers and security researchers to inspect the work instead of relying on a closed commercial service.
Ant Group has experience protecting payment and financial systems, and it says related security technology is already used in products including Alipay AI Pay and its AQ healthcare app. That background gives the project some credibility, although production use inside Ant Group does not prove the open-source framework will perform equally well in every outside environment.
The arrival of tools such as SingGuard-NSFA also exposes an uncomfortable reality about the race toward autonomous AI. Technology companies are rushing to give models access to browsers, terminals, company data, and real-world services while the industry is still trying to determine how to stop those models from being manipulated.
Open sourcing a security layer is welcome. Calling the problem solved would be straight up reckless.
Support independent tech journalism
NERDS.xyz is independently owned and operated. If you enjoy my coverage of Linux, AI, hardware, cybersecurity, and tech culture, consider supporting the site on Ko-fi.
Support NERDS.xyz