Giving AI agents access to company tools and sensitive information creates a fairly obvious question. What happens when somebody convinces those agents to do something they should not?
Cisco is adding Anthropic’s Claude Mythos to AI Defense to help businesses find those weaknesses before attackers do. Announced at Cisco Live APJC, the integration will use the model to help plan simulated attacks against AI applications and agents, with controls intended to keep that testing within authorized boundaries.
The idea has merit, although the announcement leaves plenty to prove. Cisco describes how it intends to contain the testing, but its release does not provide independent results demonstrating how much better the Mythos integration is at finding exploitable flaws.
Red teaming involves testing a system by approaching it as an adversary would. Cisco says Mythos will augment its existing adaptive red teaming framework by developing contextual, multi-step attack ideas tailored to the system being examined.
That matters because an AI agent connected to business tools presents a different problem from a chatbot answering questions. Security teams need to understand what the agent can access, which actions it can perform, and whether an attacker can manipulate how it uses those permissions.
A test that catches an inappropriate answer is useful, but businesses also need to know whether an agent can be persuaded to misuse a connected tool or expose information. Testing the actual deployment is a sensible direction, provided the resulting findings are accurate and actionable.
Cisco is also setting limits on how customers can use Mythos through AI Defense. Customers will interact through a structured interface, without open-ended access to prompt the model however they please.
According to the company, testing will operate exclusively against registered targets that the customer owns and is authorized to test. The system cannot scan arbitrary internet infrastructure, and each engagement requires authentication tied cryptographically to a named enterprise operator.
Engagements will have fixed jobs, human-defined objectives, and time-limited sessions that can be halted on demand. Cisco also says testing actions will be recorded for auditing, with results available to inform subsequent attack attempts.
Those are sensible safeguards for a product designed to generate adversarial ideas. They establish boundaries and accountability, although describing a control does not demonstrate how reliably it will hold up when something unexpected happens.
There is also a difference between finding weaknesses and establishing that an agent is safe to deploy. A successful test can expose a problem worth fixing, but passing a collection of tests cannot establish that every possible attack has been covered.
For buyers, the useful evidence would include validated findings, false-positive rates, and comparisons with the testing Cisco already offers. It would also help to see how often the results lead to practical fixes, rather than another report that a security team must spend hours sorting through.
Cisco says adaptive red teaming with Claude Mythos is coming soon for select enterprises, while its technical announcement describes broader access for qualified customers as conditional, with dates, scope, and regional availability subject to change. This is an announcement of an upcoming capability, rather than a general rollout that every customer can immediately evaluate.
Alongside the security testing, Cisco announced updates to Splunk Agent Observability intended to connect AI spending with the quality of the work agents produce. Its Tokenomics capability will use Amazon Bedrock and Amazon CloudWatch usage data to pair agent costs with evaluations of output quality and impact.
That addresses a reasonable concern. An agent completing a task cheaply is not much of a bargain if somebody must redo its work, although an evaluation score should not automatically be treated as proof of business value.
Cisco also claims its Eval-G1 evaluation models can run evaluations and guardrails at up to 98 percent lower cost than a commercial large language model, with response times 2.5 times faster than traditional LLMs. Those figures need testing context before buyers can judge how well they apply to their own workloads, including which models were compared and what accuracy was maintained.
Tokenomics is scheduled for private preview in December, with general availability through AWS Marketplace expected in early 2027. Cisco says the evaluation models and Evaluation Tuner are generally available starting this month.
The larger announcement packages security testing, agent evaluation, spending oversight, and infrastructure exposure tracking into a single enterprise pitch. There is practical value in connecting those functions, but putting them together does not itself prove that an organization’s agents are trustworthy.
Cisco is tackling a real problem, and testing agents before handing them meaningful access is a reasonable place to start. The next step is showing that Mythos consistently finds consequential weaknesses, helps customers fix them, and stays within the boundaries Cisco has promised.
Support independent tech journalism
NERDS.xyz is independently owned and operated. If you enjoy my coverage of Linux, AI, hardware, cybersecurity, and tech culture, consider supporting the site on Ko-fi.
Support NERDS.xyz
