Auto-Triage Workflow: Bug-Monitoring Agent that Connects Related Reports and Opens Fix PRs¶
An agent that monitors alerts, correlates against prior incidents, investigates, and opens a fix PR — only safe under three named preconditions.
Auto-triage is a bug-monitoring agent wired to alert streams. It watches a stream of webhooks (Sentry, Datadog, Linear, Slack, GitHub), groups related reports against memory of prior investigations, investigates with codebase and observability tools, and either tags the right human owner or opens a fix PR (Cognition: Introducing Auto-Triage, 2026-05-18).
Why auto-triage¶
Alerting systems "usually stop at detection: they create a message, ticket, or alert, and then a human has to reconstruct the context around it" (Cognition: Introducing Auto-Triage). Slack describes manual triage of flaky-test failures as a reason to automate their detection and suppression (Slack Engineering: Handling Flaky Tests at Scale). A human moving through monitor → correlate → investigate → propose-fix pays a fresh context-loading cost at each transition; auto-triage collapses that chain into one agent context.
The workflow is worth adopting only when all three of the following preconditions hold:
- A failure classifier sits upstream of dispatch. Architecture changes, database migrations, security-sensitive code, and ambiguous business logic are outside the safe scope of autonomous fix generation; an un-classified stream produces
sleeppatches around races and retry wrappers around real outages. - A confidence floor exists on the propose-fix branch. Cognition's contract relies on the agent self-assessing that "the fix is clear" (Cognition: Introducing Auto-Triage), yet Devin "does not reliably identify or prevent security vulnerabilities" (Idlen: Devin Review 2026). Without an external gate, the PR branch over-fires.
- Reviewer attention is not at capacity. 30 of 32 successfully merged AI fix PRs depended on actionable review loops (arXiv:2602.19441); reviewers who rubber-stamp the output collapse the workflow's final gate and ship agent-judgement code unsupervised.
If any one is missing, the workflow transfers diagnostic cost to reviewers instead of removing it.
Four implementation stages¶
flowchart TD
A[Alert stream] --> B[Stage 1: Monitor]
B --> C[Stage 2: Correlate]
C -->|Known issue| K[Link to prior thread]
C -->|New incident| D[Stage 3: Investigate]
D --> E[Codebase read]
D --> F[Observability query]
D --> G[Ticket search]
D --> H[Parallel sub-investigations]
E & F & G & H --> I[Stage 4: Propose-Fix]
I -->|Fix is clear AND confidence floor passed| J[Open PR]
I -->|Otherwise| L[Tag owner]
The four stages are not unique to Cognition. The same decomposition appears as perception → reasoning → planning → action in an independent LLM incident-response agent (arXiv:2602.13156); a survey of LLM-driven bug-tracking workflows expands the list to seven stages but the four are its core (arXiv:2510.08005). Replit announced a Monitoring Agent on 2026-10-22 that investigates an alert, identifies the cause, validates a fix, and deploys it to production (Replit: Monitoring Agent). Vercel's alert validation covers investigation and notification filtering. With validation enabled for a webhook destination, Vercel Agent investigates alerts "before they reach your paging system", and only the ones it classifies as Critical reach that destination (Vercel: Vercel Agent can now validate alerts before paging you). The investigate step is narrow: "Agent checks logs and metrics to assess impact."
Stage 1: Monitor¶
Auto-triage subscribes to event sources rather than polling: Cognition names Slack messages, Linear events, GitHub activity, schedules, and incoming webhooks (Cognition: Introducing Auto-Triage). The agent does not own the alert thresholds — those live in the upstream system (Sentry rule, Datadog monitor), the responsibility of the failure classifier precondition above.
Stage 2: Correlate¶
The agent groups related reports by querying long-running memory of prior investigations: "If a known issue fires new alerts, Devin can connect that to the earlier thread, de-duplicating incidents and saving significant triage time" (Cognition: Introducing Auto-Triage). The mechanism is memory-based, not rule-based — there is no per-incident fingerprint hash, no clustering algorithm in the published description.
This stage carries the oracle-poisoning surface. With no published expiry policy on the memory store, a single mislabeled dedupe compounds across months of correlated alerts, so correlation drifts toward the agent's mistakes unless the memory is bounded or rotated.
Stage 3: Investigate¶
The agent investigates with a fixed tool palette: codebase inspection (repo read), observability queries, ticket/thread search, and "sub-Devins to investigate in parallel" (Cognition: Introducing Auto-Triage). The parallel sub-agent dispatch is the load-bearing speed mechanism — sequential investigation across three observability systems exceeds the context budget before the response window closes.
The investigation context that survives this stage feeds propose-fix without re-fetching — the workflow's actual cost saving is that context-loading happens once across the chain rather than once per stage.
Stage 4: Propose-Fix¶
The agent opens a PR when "the fix is clear", or tags the right owner when not (Cognition: Introducing Auto-Triage). The decision is binary — no confidence score is surfaced, no per-incident-class threshold documented. Missing any one precondition turns this branch into autonomous PR generation against alerts the agent does not understand.
Adjacent failure-rate data bounds the expected error rate: 15.3% of unmerged AI fix PRs were closed for "incorrect or incomplete fixes" and 18.1% for introducing new test failures (arXiv:2602.00164). The agent circuit breaker pattern provides the per-fingerprint retry budget that prevents stacked low-confidence fixes against the same alert.
Triggers and constraints¶
The workflow is push-driven — the agent runs on each inbound event, not on a schedule. The constraint surface differs by stage:
| Stage | Authority bound by |
|---|---|
| Monitor | Upstream alerting rule (Sentry rule, Datadog monitor); no agent authority over thresholds |
| Correlate | Long-running memory store; should be bounded by TTL and rotated on detected miscorrelations |
| Investigate | Read-only tool palette (codebase, observability, tickets) — no side effects |
| Propose-Fix | PR creation against a constrained class list; tag-owner is the fallback when classification or confidence fails |
Side-effecting authority is concentrated in stage 4, which makes it the safest place to add a control — partition out-of-scope alert classes, require an external confidence evaluator, or follow the Seer default below — without redesigning the workflow.
Vercel's Observability Plus includes ten Agent investigations per billing cycle on Pro and Enterprise plans. Additional investigations are billed separately; this allowance is not an execution cap (Vercel: Vercel Agent can now validate alerts before paging you).
Two published defaults for proposing fixes¶
The shape has converged across vendors, but the default posture on stage 4 has not. Two production implementations ship opposite defaults:
| Vendor | Default behavior at stage 4 | Justification given |
|---|---|---|
| Devin Auto-Triage | Opens a PR when the agent self-assesses "the fix is clear" (Cognition: Introducing Auto-Triage) | Memory-driven correlation and prior-fix patterns are enough signal; human-tag branch is the escape valve when not. |
| Sentry Seer | No PR without explicit user prompt; PR creation can be disabled globally; code generation can be delegated to Claude Code or Cursor Cloud Agents (Sentry Docs: Seer) | Investigation is high-value and low-risk because it has no side effects; the propose-fix stage transfers cognitive cost to reviewers without removing it unless gated explicitly. |
A reasonable practitioner can defend stopping at stage 3 entirely — deliver an investigation summary tagged to the right owner and let the human decide whether the fix is mechanical enough to delegate to a separate coding agent. This Seer default is not a degraded form of the workflow; it is the conservative choice when stage 4's preconditions cannot be met.
Multi-tool coverage¶
The four-stage shape is tool-agnostic — Cognition's Devin Auto-Triage and Sentry's Seer both ship it, and the underlying ReAct-style perception/reasoning/planning/action decomposition is vendor-independent (arXiv:2602.13156). Tool choice matters at stage 4: Cognition's harness opens the PR itself, while Seer can delegate code generation to Claude Code or Cursor Cloud Agents (Sentry Docs: Seer) — the cleanest way to keep the investigate stage in the alerting platform while moving propose-fix into a coding-agent harness with separate retry and review controls.
Why it works¶
Auto-triage produces faster recovery than serial human triage because it collapses four cognitive context-switches into one agent context. An agent that holds investigation context across the chain dispatches sub-investigations in parallel and re-uses telemetry already fetched in the correlate stage — the four-stage decomposition demonstrates 23% faster recovery than frontier-LLM baselines specifically because in-context refinement avoids redundant retrieval (arXiv:2602.13156). Cognition's mechanism is the same effect: known-issue alerts short-circuit to known patches, letting stage 2 hand stage 4 a pre-validated patch context that would otherwise cost a full investigation cycle (Cognition: Introducing Auto-Triage).
When this backfires¶
- Upstream alert quality is poor. Auto-triage inherits the platform's grouping; dedupe-by-memory cannot fix a stream where one flake emits many distinct fingerprints, nor separate one where distinct bugs share a fingerprint. Slack Engineering separates flaky-test handling from other test failures. Keep that distinction in the dispatch decision.
- Reviewer attention is saturated. Empirically, agent-PR merge success correlates with actionable review loops (arXiv:2602.19441). Teams where reviewers are at capacity will rubber-stamp the auto-triage output, collapsing the workflow's final gate; the design then ships agent-judgement code unsupervised at scale.
- The agent misses security risks. Cognition's escalation contract assumes the agent can identify a clear fix, but the Idlen review reports that Devin "does not reliably identify or prevent security vulnerabilities" (Idlen: Devin Review 2026). Without an external confidence floor — a separate classifier that rejects low-signal investigations before they reach stage 4 — the propose-fix branch over-fires.
- High-blast-radius alert classes are not partitioned out. Architecture changes, database migrations, security-sensitive code, ambiguous business logic, and zero-day attacks fall outside the safe scope of autonomous fix generation (Idlen review; Panther: AI Alert Triage Automation). Dispatching these without a per-incident-class gate is unsafe.
- Long-lived memory drifts. The dedupe-by-memory mechanism has no published expiry policy (Cognition: Introducing Auto-Triage). A single mislabeled dedupe compounds across months of correlated alerts unless the memory store is bounded or rotated explicitly.
Example¶
The minimal four-stage trigger contract for a Sentry → auto-triage → fix-PR loop, with the three preconditions surfaced as explicit dispatcher controls:
# Auto-triage dispatcher contract (Sentry → bug-monitoring agent)
on:
sentry_event:
types: [issue_alert]
preconditions:
# Precondition 1: failure classifier upstream of dispatch
classify:
flake_detector: sentry-flake-rule@v2
in_scope_classes: [null_pointer, type_error, missing_import, simple_logic]
out_of_scope_classes: [migration, auth, infrastructure, business_logic]
# Precondition 2: confidence floor on the propose-fix branch
confidence:
min_score_for_pr: 0.85 # below this, escalate (tag owner only)
require_external_evaluator: true
# Precondition 3: reviewer-budget circuit breaker
retry_budget:
max_attempts_per_fingerprint: 1
cooldown_minutes: 60
stages:
monitor: subscribe(sentry, datadog, github)
correlate:
memory_ttl_days: 30 # bound the oracle-poisoning surface
rotate_on_miscorrelation: true
investigate:
tools: [repo_read, logs_query, traces_query, ticket_search]
parallel_subagents: 3
propose_fix:
output: [pr, owner_tag] # both branches enumerated
no_pr_classes: [migration, auth, security]
The contract makes the three preconditions executable rather than implicit — each is a named field with a value that fails closed if missing. The out_of_scope_classes list and the no_pr_classes list overlap deliberately: a class is either fully ineligible for dispatch (classifier rejects) or eligible only for stage-3 output (investigation summary, owner tag, no PR).
Key Takeaways¶
- The four-stage decomposition — monitor, correlate, investigate, propose-fix — has converged across Cognition's Devin Auto-Triage, Sentry's Seer, and independent academic LLM incident-response agents; the shape is canonical, not vendor-specific.
- Stage 2's dedupe-by-memory is the load-bearing speed mechanism but also the oracle-poisoning surface — bound the memory store's TTL and rotate on miscorrelation.
- Stage 4 has two published defaults; Cognition opens a PR when "the fix is clear", Seer requires explicit prompt. The conservative default (stop at stage 3, hand off to a separate coding agent downstream) is not a degraded form of the workflow.
- The workflow is only safe when three preconditions hold: a failure classifier upstream of dispatch, a confidence floor on the propose-fix branch, and reviewer attention that is not at capacity. Missing any one transfers cost to reviewers instead of removing it.
Related¶
- Agent Circuit Breaker — the per-fingerprint retry budget pattern that prevents stacked low-confidence fixes against the same alert
- Agent-Gated Pager Escalation: Judged Criticality on the Notification Destination — the gate upstream of stage 1, where an agent decides whether an alert reaches a human at all
- Incident Log Investigation Skill: Parallel Queries — the stage-3 investigate sub-pattern in isolation, with parallel observability queries and eval-backed precision
- Closed-Loop CI Failure Remediation with Cloud Coding Agents — the propose-fix branch's three required preconditions documented for the adjacent CI-failure trigger
- Continuous AI: A Navigation Map of Always-On Agent Workflows — the parent map placing this in the triage family alongside the continuous-* workflows