AI Agent Incident Severity Matrix: Classify Impact, Escalate Safely, Preserve Evidence

A customer-success lead sees two of the same automated update in a customer thread. The agent’s run shows a timeout, the downstream state is unclear, and somebody is asking whether to retry. The first useful decision is not “how do we fix it?” It is “how serious is this, who has authority to contain it, and what evidence must we preserve before the next action?”

Use this when: an AI worker may have sent, changed, published, exposed, deleted, or otherwise attempted a real-world action and the team needs a shared severity call in minutes. It is a classification tool for the incident owner, not a recovery runbook or a score for the team.

Make the call part of the operating path: open one incident ticket with the affected workflow, known scope, action state, evidence links, and next reassessment time; route the decision to the accountable owner; then keep containment and customer communications connected to that record. In Midpoint, people and AI workers can coordinate that handoff in channels while tickets carry the durable decision and workflows remain inside the controls the team has set.

This guide owns classification criteria, impact thresholds, escalation triggers, and evidence requirements. Use the AI agent failure recovery playbook for containment, reconciliation, rollback, and repair mechanics. Use an AI agent incident communication plan for stakeholder messages and update cadence. Use an AI workflow audit trail for durable accountability across normal work.

Why an AI agent needs a different severity lens

Traditional availability severity is necessary but incomplete for an agent. An API returning errors can be disruptive; an agent that successfully performs the wrong side effect can be worse. Classifying only by uptime misses four signals that should change the decision:

• External action: Did the agent send, change, submit, purchase, delete, publish, or otherwise affect a real party or system?
• Action certainty: Can the team prove which individual actions completed, failed, or remain unknown?
• Exposure: Does the event involve sensitive data, credentials, regulated work, a security boundary, money, safety, or a contractual commitment?
• Blast radius: How many users, records, workflows, or downstream teams are affected now, and how quickly could that number grow?

Google SRE guidance treats severity as an impact-based triage decision and recommends making the highest-severity decision criteria finite enough for an on-call responder to apply under pressure. That is a useful starting point. For AI workflows, add the explicit question: was an irreversible or high-consequence action attempted, and can its final state be proven?

The matrix is deliberately conservative about unknown state. Unknown is not proof of harm, but it is a reason to avoid automatic replay and to raise the decision to an owner who can balance interruption against duplicate action risk. OpenAI’s agent guidance makes the related engineering point: validation and approval should sit next to the tool that causes the side effect. Classification is the operating layer that decides when those controls may no longer be sufficient on their own.

The severity matrix

Evaluate each incident against the highest applicable severity tier. Do not average impact dimensions: a suspected sensitive-data disclosure can be a SEV 1 even if the affected population is initially small. The thresholds are a starting operating policy, not a substitute for legal, security, safety, or contractual obligations.

SEV 1 - Critical

• User and business impact: Widespread outage, material customer harm, irreversible or high-value external effect, or a critical workflow unavailable without a viable workaround
• Agent behavior and state: Confirmed or plausible unsafe action; action state unknown for consequential work; control bypass; or repeated execution is expanding impact
• Exposure and growth: Suspected security or sensitive-data event; regulatory, safety, financial, or contractual exposure; rapid blast-radius growth
• Required decision: Immediately declare the incident, freeze the affected capability when safe, engage the accountable incident, security/risk, and business owners, and preserve evidence before destructive changes.

SEV 2 - Major

• User and business impact: Material degradation for a defined customer group or business process; repeated inaccurate outputs or external actions with limited but real impact
• Agent behavior and state: An agent made incorrect, duplicate, or unauthorized changes that are bounded; or a required tool/approval path is failing for a significant workflow
• Exposure and growth: Moderate exposure, growing queue, or scope that cannot yet be fully reconciled
• Required decision: Escalate to the responsible service and business owners promptly; prevent further expansion; set a short reassessment time.

SEV 3 - Moderate

• User and business impact: Limited, reversible effect on a small group; usable workaround exists; internal operations are disrupted but customers are not materially blocked
• Agent behavior and state: Isolated failed run, incorrect output caught before a side effect, or bounded error with known completion state
• Exposure and growth: No known sensitive-data, security, safety, or high-value impact; low likelihood of expansion
• Required decision: Record the event, assign an owner, investigate during the agreed response window, and automatically reassess if a trigger is crossed.

SEV 4 - Low / signal

• User and business impact: No confirmed user impact or only a test/sandbox effect
• Agent behavior and state: Alert, evaluation regression, guardrail rejection, near miss, or a defect caught before production side effects
• Exposure and growth: No credible exposure and no expanding pattern
• Required decision: Log and triage. Promote if evidence shows a real production effect or repeated near misses.

The category names are less important than their operating consequences. Atlassian also frames severity levels around customer impact and notes that a documented, agreed matrix produces more consistent response decisions. Keep the matrix short enough to apply in minutes, then anchor it to concrete examples from your own workflows.

A practical decision tree for AI workflow severity levels

Use this decision tree at first credible signal. It is intentionally about classification, not remediation steps.

1. Is there a credible safety, security, sensitive-data, legal, financial, or irreversible-action concern?
   • Yes, or the answer cannot be established quickly for a high-consequence action: classify SEV 1 pending evidence.
   • No: continue.
2. Did the agent create or attempt an external side effect?
   • Yes: identify the affected object population and whether each action is confirmed, failed, or unknown.
   • No: continue to impact assessment.
3. Is action state unknown for anything consequential?
   • Yes: classify at least SEV 2. Promote to SEV 1 if the object is high value, sensitive, irreversible, or the blast radius can grow through retries.
   • No: continue.
4. Are users or a material business workflow currently blocked, receiving wrong results, or being affected repeatedly?
   • Broad or material impact: SEV 1 or SEV 2 depending on exposure and reversibility.
   • Limited, reversible impact with a workaround: SEV 3.
   • No confirmed impact: SEV 4.
5. Has the condition crossed a promotion trigger?
   • If yes, reclassify and record the time, trigger, and deciding owner. Do not wait for a scheduled review.

This decision tree is designed to reduce two common classification mistakes. The first is treating a model-quality problem as harmless because infrastructure is available. The second is classifying a visible outage as the only serious event while an unknown external action is still being retried in the background.

Define thresholds before the incident

A matrix becomes operational only when teams replace vague words with boundaries. “Many users,” “sensitive,” and “soon” are not thresholds. Write down values that fit the workflow, then review them when scope or contracts change.

For each agent, define:

• Population thresholds: a count or percentage of customers, accounts, tasks, funds, records, or critical queues that turns SEV 3 into SEV 2 and SEV 2 into SEV 1.
• Time thresholds: the maximum allowed duration for a blocked critical workflow, a growing queue, or an unreviewed unknown-action state.
• Consequence thresholds: action types that are SEV 1 by default, such as changing permissions, moving money, deleting records, sending regulated notices, publishing content, or accessing restricted data.
• Confidence thresholds: what evidence is needed to call an action completed, failed, or unknown. A timeout is not a failure result. A tool acknowledgement is not always end-state proof.
• Pattern thresholds: repeated lower-severity events, near misses, or guardrail rejections that warrant promotion because they indicate a systematic control gap.

A useful policy distinguishes severity from priority. Severity expresses current impact and exposure. Priority expresses how much delivery work should be scheduled after the incident. A SEV 3 defect can be high priority to prevent recurrence; a SEV 1 classification should never be delayed because somebody has not assigned a roadmap priority.

NIST’s AI RMF Playbook supports this discipline by calling for processes to document reported errors, near misses, incidents, negative impacts, impact/severity assessments, and responses. That is a governance requirement you can make practical: one incident record should show which threshold was crossed, who made the call, and what evidence supported it.

Escalation triggers that should override the original label

No initial label is permanent. Set explicit triggers that promote severity without debate. At a minimum, include these:

• A suspected security, privacy, or sensitive-data boundary is crossed. Promote to SEV 1 and involve the appropriate security or privacy authority.
• An action becomes unknown after a high-consequence call. Promote to at least SEV 2. Treat replay as a separate decision until reconciliation evidence exists.
• A side effect repeats or accelerates. Duplicate sends, repeated record changes, or expanding tool calls are a blast-radius trigger, even if each individual event looks small.
• A workaround fails or the critical-workflow duration threshold is exceeded. Promote based on real user or business effect, not server metrics alone.
• A human approval, policy check, or guardrail was bypassed. Promote to SEV 1 or SEV 2 based on action consequence and exposure, then preserve the policy and trace evidence.
• The known scope materially changes. A small initial sample may become a broad population after log analysis; record both scope estimates and reclassify.
• A credible external report contradicts internal telemetry. Investigate the discrepancy instead of using incomplete telemetry to hold the lower label.

For every escalation, capture the prior severity, new severity, trigger, decision time in UTC, and accountable decision-maker. This prevents a post-incident argument over whether a team “should have known” from becoming a search through chat history.

Evidence requirements by severity

Evidence is not a bureaucratic appendix. It is what makes impact claims, escalation decisions, and later reconciliation defensible. Collect enough to preserve facts, but minimize personal data and access to sensitive artifacts.

Detection source and UTC timestamp

• SEV 1: Required
• SEV 2: Required
• SEV 3: Required
• SEV 4: Required

Agent/workflow version, model/configuration, tool and policy context

• SEV 1: Required
• SEV 2: Required
• SEV 3: Required where relevant
• SEV 4: Recommended

Population definition and affected-object identifiers

• SEV 1: Required, with protected handling
• SEV 2: Required
• SEV 3: Required if external effect
• SEV 4: Summary only

Per-action state: completed, failed, or unknown

• SEV 1: Required
• SEV 2: Required for side effects
• SEV 3: Required if applicable
• SEV 4: Not normally needed

Severity rationale and threshold crossed

• SEV 1: Required
• SEV 2: Required
• SEV 3: Required
• SEV 4: Required for promotion

Decision and approval trail

• SEV 1: Required
• SEV 2: Required where external impact or exception exists
• SEV 3: Recommended
• SEV 4: Optional

Preserved trace/log references and integrity controls

• SEV 1: Required
• SEV 2: Required
• SEV 3: Recommended
• SEV 4: Recommended

Validation and reconciliation result

• SEV 1: Required before close
• SEV 2: Required before close if side effect occurred
• SEV 3: Required when needed
• SEV 4: N/A

A few rules make this usable. Store references to protected logs rather than copying sensitive payloads into wide-access incident notes. Use immutable identifiers for actions, messages, workflows, and approval requests. Mark uncertainty explicitly. If the team does not know whether an action completed, the evidence record should say unknown, not “probably failed.”

The audit trail and the incident packet serve different purposes. The audit trail records accountable workflow activity over time. The incident packet is a curated, time-bounded evidence bundle that explains a classification and its impact. The AI agent handoff patterns guide is the companion for changing ownership: hand off the current severity, known scope, evidence location, open decision, and next reassessment time, not a vague summary.

Example: a duplicate customer-action incident

An outreach agent times out after requesting a send. Its retry system starts another attempt. The API dashboard later shows two sends for a subset of contacts, but the team does not yet know whether the timeout caused only duplicates or also partial delivery failures.

The first timeout alone might be SEV 3 if no side effect is known. The moment evidence suggests an external duplicate and the action state is unknown for other contacts, it becomes at least SEV 2: there is a defined affected population, a customer-facing effect, and a risk of expansion through retries. If the message contains regulated content, affects a broad audience, or continues to send, it becomes SEV 1.

The severity record should capture the retry policy version, original action identifiers, duplicate evidence, affected-population query, timestamp of the promotion, and the owner who froze further sends. The recovery playbook then handles reconciliation and safe compensation. The communication plan handles who tells affected customers what. The severity matrix has already done its job: it got the right authority and evidence standard involved before the incident expanded.

Put the matrix into production

Start with the four-level matrix, then run one short exercise per high-consequence workflow. Give the team an ambiguous scenario: an agent approved a tool call but the downstream service timed out; a retrieval source produced unsafe instructions; a batch began updating records before the validation alarm fired. Have the team classify it in five minutes, identify the promotion trigger, and list the evidence they would need to change the label.

Review the results quarterly and after meaningful incidents. If responders disagree consistently, the matrix is underspecified. If every event becomes SEV 1, the thresholds are not separating urgent exposure from normal defect work. A good matrix makes the first decision faster and more legible without pretending that a matrix can replace judgment.

Teams using Midpoint Enterprise can coordinate people, AI workers, approvals, and evidence in one visible operating model. The real operational control is still the policy you make explicit: which impacts matter, who can elevate a decision, and what proof is required before the team declares an agent incident understood.

Sources

• Google SRE: Incident Management Guide
• Google SRE Workbook: Being On-Call
• Atlassian: Understanding Incident Severity Levels
• Atlassian: Create an Impact, Urgency, Priority Matrix
• NIST AI RMF Playbook: Manage
• OpenAI: Guardrails and Human Review