Every security team running AI agents eventually gets the same question from the board, from an auditor, or from their own late-night worries: "Are we sure this thing can't be talked into something stupid?"

You can show them dashboards. You can show them policies, approvals and a tamper-evident audit trail of every action your agents have taken. And all of that is valuable. But it answers a different question. It tells you what your agents did.

It doesn't tell you what somebody could make them do.

That gap is exactly why we built Red Teaming into MITRITY: not as a one-off pentest you commission twice a year, and not as a pile of scary prompts dumped in a spreadsheet, but as a governed, scheduled, budgeted part of the same platform that already governs your agents.

Attacking your own agents, on purpose

The idea behind red teaming is old and simple: before someone else attacks your system, you do it yourself, under controlled conditions, and you fix what breaks.

With AI agents, that simple idea gets complicated quickly. The attack surface isn't a port or an endpoint. It's a conversation. An agent can leak its system prompt, drift off its mission, or misuse a tool it legitimately holds, all without a single policy being violated. Those failures happen inside the agent's own reasoning, where traditional controls can't see.

So you have to talk to it. Persistently, creatively and at scale.

Campaigns, oracles and findings

Red Teaming in MITRITY is built around three ideas:

A campaign sends attempts at a target you choose.
An oracle decides which of those attempts actually got somewhere.
A finding is what survives: a severity, a description of what happened, a remediation, and enough information to reproduce it.

A campaign can point at two kinds of targets today:

An LLM route behind the Mitrity LLM Gateway. Here you're testing your guardrails: the prompts going in, the responses coming out and the policies sitting between them.
Your own agent, reached over HTTP one turn at a time. Here you're testing the agent itself. Will it disclose its configuration? Can it be talked off its mission? Will it reach for a tool it shouldn't? You give the campaign an endpoint, a JSON request template and a path to the reply text, and the runner takes it from there. Each answer is scored before the next turn is sent, so a multi-turn session stops the moment something gives, and the finding tells you exactly which turn it happened on.

One detail we're quietly proud of: a campaign never carries a credential. The runner presents its own, from its own environment, and the target host must be on an allowlist set where the runner is deployed. A campaign can pick the endpoint. It cannot pick a host you didn't allow.

Safe by default, because attacks are real

Here's the uncomfortable truth about red teaming: you are deliberately pointing attacks at your own systems. If the tooling isn't careful, the test becomes the incident.

So we made caution the default, not a setting you have to remember.

Every new campaign starts in dry run. Each attempt is selected, governed, costed and scored, but nothing reaches the target. You see exactly what a run would have sent, what your governance would have done with it and what it would have cost, without touching a live system.
Leaving dry run is a deliberate act. There's no toggle and no bulk action. You type the campaign's name to confirm, and the API checks it again on its side. If the target environment is production, you get a stronger warning first, and the runner has its own second switch on top. Our advice is simple: point agent campaigns at staging.
Attacks don't get a free pass. Every attempt goes through the same decision engine as any other action from that environment. If your policies block it, it's blocked and logged. That's not an obstacle to the test. A blocked attempt is a result: it's proof your governance held.
Every run has hard ceilings. An attempt limit, a time limit, a token budget and a spend budget. If the target starts erroring above the threshold you set, the run stops instead of pushing a struggling system over the edge. And pausing a campaign stops the current run at its next attempt, no confirmation needed.
Unsent is not the same as blocked. When a run stops early, the attempts it never sent are counted as not attempted, never folded into blocked. That sounds like a small accounting detail. It isn't. Counting unsent attempts as blocked would turn a truncated run into a clean bill of health, and that's the kind of quiet lie a security tool should never tell you.

Proved, matched or argued: knowing how sure you are

Not all findings are created equal, and pretending otherwise is how teams end up either ignoring everything or panicking about everything.

In MITRITY, every oracle carries a confidence level, and that confidence travels with every finding it produces:

Deterministic. The outcome is a fact. A planted canary marker showed up where it must never appear. Content that should have been redacted wasn't. The gateway recorded a specific governance decision. There's nothing to debate.
Heuristic. A pattern matched in the reply, or a tool call outside the agent's mission scope showed up on the audit trail. Usually right, but a pattern can be wrong.
Judged. A model assessed the exchange. These are real findings and worth reading, but they're an argument, not a proof. Weigh them before acting on them at scale.

The practical upshot: a queue of critical findings is never a wall of equally scary red. You can see at a glance which ones were proved and which ones were argued, and spend your morning accordingly.

The attack library you never have to hold

Attempts are drawn from a corpus of attacks that MITRITY maintains and versions. When you build a campaign, you choose which classes of attack it draws from. You never author, read or store the attacks themselves, and neither does your control plane.

That's deliberate. A library of working attacks against AI systems is a liability to store, a liability to review and a liability to leak. Keeping it out of your tenant means there's nothing in your account for an attacker to steal, and nothing for an auditor to ask you to justify holding.

But you still need to reproduce what went wrong. So every finding carries a reproduction key: the corpus entry identifier, the corpus version, the mutation applied and the seed. Those four values replay the attempt exactly. None of them tells anyone how to write the attack.

What stays home

The runner executes inside your environment and sees the whole exchange: the full prompt, the full response and every turn of a multi-turn attempt. All of that stays in your own logs, under your own retention.

What comes back to the control plane is intentionally thin:

SHA-256 hashes of the prompt and response, so you can correlate a finding with your logs and prove two attempts were identical, without revealing content.
A redacted excerpt of at most 512 characters. Redaction happens in your environment before anything is sent, and if redaction can't be performed, nothing is sent. The finding simply says the excerpt was omitted.
The metadata: outcome, oracle, confidence, the turn reached, the reproduction key, counts and costs.

When you need the full transcript, it's exactly where it has always been: with you.

From finding to fix, and proof that it stuck

A red team report that ends with "here are 40 problems, good luck" isn't much of a gift. We wanted the loop to close inside the platform.

One weakness is one row. If the same weakness shows up again, against the same target from the same corpus entry, the existing finding is updated rather than duplicated. Its occurrence count goes up and its last-seen time moves forward. Your queue doesn't grow just because your schedule ran again.
Some findings come with a fix attached. When the canary oracle proves a leak, MITRITY can propose a detection pattern derived from what the attempt planted, never from the attack text. Proposing opens a governance review. Approving it adds the pattern to your DLP library, and nothing is enforced until you bind it to a policy yourself.
Findings close on evidence, not on optimism. Approving the review doesn't mark anything resolved. The finding closes when a later real run of the same campaign shows the fix working: either the enforcement point refused the attempt, or the runner planted its marker and none of it came back. If the weakness reappears later, the finding reopens instead of staying quietly resolved.
Accepted risk stays visible. If you look at a finding and decide to live with it, you can mark it as an accepted risk. It stays visible on the compliance view, because an accepted risk is a documented decision, not a closed gap.

A score that doesn't flatter you

Each agent and campaign gets a guardrail assurance score answering one question: of the attack classes the latest real runs exercised, how many held?

A class only counts as held when the verdict is verified. Dry runs don't score. Runs that stopped early leave their classes undecided, and undecided is never counted as held. Every gap is spelled out in a caveat on the card, so a 100% with three caveats reads as exactly what it is.

Evidence your auditors will recognise

The compliance tab regroups the same findings the way an assessment asks for them, mapped to OWASP LLM Top 10, MITRE ATLAS, NIST AI RMF, ISO/IEC 42001 and the EU AI Act.

To be clear about what that is: these mappings are MITRITY's own view of which control a given attack exercises. They're not a certification. They are a fast, organised way to bring evidence to the conversation and to find the gaps worth arguing about.

Trust what you've tested, not just what you've logged

Governance tells you your agents followed the rules yesterday. Red teaming tells you whether they'll hold when somebody tries hard to make them break the rules tomorrow.

You need both. And you need the second one to be as careful, bounded and auditable as the first.

For the full technical detail on campaigns, oracles, budgets and what leaves your environment, read the Red Teaming documentation.

MITRITY is an intent-aware governance platform for autonomous AI agents. Start a free trial or read the documentation to run your first governed red team campaign, safely in dry run, today.

What Could Somebody Make Your Agent Do? Red Teaming, Governed.