Skip to content

Auto-Triage Workflow: Bug-Monitoring Agent that Connects Related Reports and Opens Fix PRs

An agent that monitors alerts, correlates against prior incidents, investigates, and opens a fix PR — only safe under three named preconditions.

Auto-triage is a bug-monitoring agent wired to alert streams. It watches a stream of webhooks (Sentry, Datadog, Linear, Slack, GitHub), groups related reports against memory of prior investigations, investigates with codebase and observability tools, and either tags the right human owner or opens a fix PR (Cognition: Introducing Auto-Triage, 2026-05-18).

Why auto-triage

Traditional alerting "usually stop[s] at detection: they create a message, ticket, or alert, and then a human has to reconstruct the context around it" (Cognition: Introducing Auto-Triage). That reconstruction has measurable cost — production SOC deployments see false-positive rates "approach 99% in some cases" (Security Boulevard: AI Alert Triage, Apr 2026), and ~25% of CI failures are flake rather than real defects (Slack Engineering: Handling Flaky Tests at Scale). A human moving through monitor → correlate → investigate → propose-fix pays a fresh context-loading cost at each transition; auto-triage collapses that chain into one agent context.

The workflow is worth adopting only when all three of the following preconditions hold:

  • A failure classifier sits upstream of dispatch. Architecture changes, database migrations, security-sensitive code, and ambiguous business logic are outside the safe scope of autonomous fix generation; an un-classified stream produces sleep patches around races and retry wrappers around real outages.
  • A confidence floor exists on the propose-fix branch. Cognition's contract relies on the agent self-assessing that "the fix is clear" (Cognition: Introducing Auto-Triage), yet Devin "does not always surface uncertainty or flag dangerous actions" (Idlen: Devin Review 2026). Without an external gate, the PR branch over-fires.
  • Reviewer attention is not at capacity. 30 of 32 successfully merged AI fix PRs depended on actionable review loops (arXiv:2602.19441); reviewers who rubber-stamp the output collapse the workflow's final gate and ship agent-judgement code unsupervised.

If any one is missing, the workflow transfers diagnostic cost to reviewers instead of removing it.

Four implementation stages

flowchart TD
    A[Alert stream] --> B[Stage 1: Monitor]
    B --> C[Stage 2: Correlate]
    C -->|Known issue| K[Link to prior thread]
    C -->|New incident| D[Stage 3: Investigate]
    D --> E[Codebase read]
    D --> F[Observability query]
    D --> G[Ticket search]
    D --> H[Parallel sub-investigations]
    E & F & G & H --> I[Stage 4: Propose-Fix]
    I -->|Fix is clear AND confidence floor passed| J[Open PR]
    I -->|Otherwise| L[Tag owner]

The four stages are not unique to Cognition. The same decomposition appears as perception → reasoning → planning → action in an independent LLM incident-response agent (arXiv:2602.13156); a survey of LLM-driven bug-tracking workflows expands the list to seven stages but the four are its core (arXiv:2510.08005).

Stage 1: Monitor

Auto-triage subscribes to event sources rather than polling: Cognition names Slack messages, Linear events, GitHub activity, schedules, and incoming webhooks (Cognition: Introducing Auto-Triage). The agent does not own the alert thresholds — those live in the upstream system (Sentry rule, Datadog monitor), the responsibility of the failure classifier precondition above.

Stage 2: Correlate

The agent groups related reports by querying long-running memory of prior investigations: "If a known issue fires new alerts, Devin can connect that to the earlier thread, de-duplicating incidents and saving significant triage time" (Cognition: Introducing Auto-Triage). The mechanism is memory-based, not rule-based — there is no per-incident fingerprint hash, no clustering algorithm in the published description.

This stage carries the oracle-poisoning surface. With no published expiry policy on the memory store, a single mislabeled dedupe compounds across months of correlated alerts, so correlation drifts toward the agent's mistakes unless the memory is bounded or rotated.

Stage 3: Investigate

The agent investigates with a fixed tool palette: codebase inspection (repo read), observability queries, ticket/thread search, and "sub-Devins to investigate in parallel" (Cognition: Introducing Auto-Triage). The parallel sub-agent dispatch is the load-bearing speed mechanism — sequential investigation across three observability systems exceeds the context budget before the response window closes.

The investigation context that survives this stage feeds propose-fix without re-fetching — the workflow's actual cost saving is that context-loading happens once across the chain rather than once per stage.

Stage 4: Propose-Fix

The agent opens a PR when "the fix is clear", or tags the right owner when not (Cognition: Introducing Auto-Triage). The decision is binary — no confidence score is surfaced, no per-incident-class threshold documented. Missing any one precondition turns this branch into autonomous PR generation against alerts the agent does not understand.

Adjacent failure-rate data bounds the expected error rate: 15.3% of unmerged AI fix PRs were closed for "incorrect or incomplete fixes" and 18.1% for introducing new test failures (arXiv:2602.00164). The agent circuit breaker pattern provides the per-fingerprint retry budget that prevents stacked low-confidence fixes against the same alert.

Triggers and constraints

The workflow is push-driven — the agent runs on each inbound event, not on a schedule. The constraint surface differs by stage:

Stage Authority bound by
Monitor Upstream alerting rule (Sentry rule, Datadog monitor); no agent authority over thresholds
Correlate Long-running memory store; should be bounded by TTL and rotated on detected miscorrelations
Investigate Read-only tool palette (codebase, observability, tickets) — no side effects
Propose-Fix PR creation against a constrained class list; tag-owner is the fallback when classification or confidence fails

Side-effecting authority is concentrated in stage 4, which makes it the safest place to add a control — partition out-of-scope alert classes, require an external confidence evaluator, or follow the Seer default below — without redesigning the workflow.

The two published defaults: Cognition vs Seer

The shape has converged across vendors, but the default posture on stage 4 has not. Two production implementations ship opposite defaults:

Vendor Default behavior at stage 4 Justification given
Devin Auto-Triage Opens a PR when the agent self-assesses "the fix is clear" (Cognition: Introducing Auto-Triage) Memory-driven correlation and prior-fix patterns are enough signal; human-tag branch is the escape valve when not.
Sentry Seer No PR without explicit user prompt; PR creation can be disabled globally; code generation can be delegated to Claude Code or Cursor Cloud Agents (Sentry Docs: Seer) Investigation is high-value and low-risk because it has no side effects; the propose-fix stage transfers cognitive cost to reviewers without removing it unless gated explicitly.

A reasonable practitioner can defend stopping at stage 3 entirely — deliver an investigation summary tagged to the right owner and let the human decide whether the fix is mechanical enough to delegate to a separate coding agent. This Seer default is not a degraded form of the workflow; it is the conservative choice when stage 4's preconditions cannot be met.

Multi-tool coverage

The four-stage shape is tool-agnostic — Cognition's Devin Auto-Triage and Sentry's Seer both ship it, and the underlying ReAct-style perception/reasoning/planning/action decomposition is vendor-independent (arXiv:2602.13156). Tool choice matters at stage 4: Cognition's harness opens the PR itself, while Seer can delegate code generation to Claude Code or Cursor Cloud Agents (Sentry Docs: Seer) — the cleanest way to keep the investigate stage in the alerting platform while moving propose-fix into a coding-agent harness with separate retry and review controls.

Why it works

Auto-triage produces faster recovery than serial human triage because it collapses four cognitive context-switches into one agent context. An agent that holds investigation context across the chain dispatches sub-investigations in parallel and re-uses telemetry already fetched in the correlate stage — the four-stage decomposition demonstrates 23% faster recovery than frontier-LLM baselines specifically because in-context refinement avoids redundant retrieval (arXiv:2602.13156). Cognition's mechanism is the same effect: known-issue alerts short-circuit to known patches, letting stage 2 hand stage 4 a pre-validated patch context that would otherwise cost a full investigation cycle (Cognition: Introducing Auto-Triage).

When this backfires

  • Upstream alert quality is poor. Auto-triage inherits the platform's grouping; dedupe-by-memory cannot fix a stream where one flake emits many distinct fingerprints, nor separate one where distinct bugs share a fingerprint. Per Slack Engineering, roughly a quarter of CI failures are flake — an un-classified stream yields sleep patches and retry wrappers shaped like fixes.
  • Reviewer attention is saturated. Empirically, agent-PR merge success correlates with actionable review loops (arXiv:2602.19441). Teams where reviewers are at capacity will rubber-stamp the auto-triage output, collapsing the workflow's final gate; the design then ships agent-judgement code unsupervised at scale.
  • The agent does not surface uncertainty. Cognition's escalation contract assumes the agent self-assesses confidence accurately, but Devin specifically "does not always surface uncertainty or flag dangerous actions" (Idlen: Devin Review 2026). Without an external confidence floor — a separate classifier that rejects low-signal investigations before they reach stage 4 — the propose-fix branch over-fires.
  • High-blast-radius alert classes are not partitioned out. Architecture changes, database migrations, security-sensitive code, ambiguous business logic, and zero-day attacks fall outside the safe scope of autonomous fix generation (Idlen review; Panther: AI Alert Triage Automation). Dispatching these without a per-incident-class gate is unsafe.
  • Long-lived memory drifts. The dedupe-by-memory mechanism has no published expiry policy (Cognition: Introducing Auto-Triage). A single mislabeled dedupe compounds across months of correlated alerts unless the memory store is bounded or rotated explicitly.

Example

The minimal four-stage trigger contract for a Sentry → auto-triage → fix-PR loop, with the three preconditions surfaced as explicit dispatcher controls:

# Auto-triage dispatcher contract (Sentry → bug-monitoring agent)
on:
  sentry_event:
    types: [issue_alert]

preconditions:
  # Precondition 1: failure classifier upstream of dispatch
  classify:
    flake_detector: sentry-flake-rule@v2
    in_scope_classes: [null_pointer, type_error, missing_import, simple_logic]
    out_of_scope_classes: [migration, auth, infrastructure, business_logic]
  # Precondition 2: confidence floor on the propose-fix branch
  confidence:
    min_score_for_pr: 0.85       # below this, escalate (tag owner only)
    require_external_evaluator: true
  # Precondition 3: reviewer-budget circuit breaker
  retry_budget:
    max_attempts_per_fingerprint: 1
    cooldown_minutes: 60

stages:
  monitor: subscribe(sentry, datadog, github)
  correlate:
    memory_ttl_days: 30          # bound the oracle-poisoning surface
    rotate_on_miscorrelation: true
  investigate:
    tools: [repo_read, logs_query, traces_query, ticket_search]
    parallel_subagents: 3
  propose_fix:
    output: [pr, owner_tag]      # both branches enumerated
    no_pr_classes: [migration, auth, security]

The contract makes the three preconditions executable rather than implicit — each is a named field with a value that fails closed if missing. The out_of_scope_classes list and the no_pr_classes list overlap deliberately: a class is either fully ineligible for dispatch (classifier rejects) or eligible only for stage-3 output (investigation summary, owner tag, no PR).

Key Takeaways

  • The four-stage decomposition — monitor, correlate, investigate, propose-fix — has converged across Cognition's Devin Auto-Triage, Sentry's Seer, and independent academic LLM incident-response agents; the shape is canonical, not vendor-specific.
  • Stage 2's dedupe-by-memory is the load-bearing speed mechanism but also the oracle-poisoning surface — bound the memory store's TTL and rotate on miscorrelation.
  • Stage 4 has two published defaults; Cognition opens a PR when "the fix is clear", Seer requires explicit prompt. The conservative default (stop at stage 3, hand off to a separate coding agent downstream) is not a degraded form of the workflow.
  • The workflow is only safe when three preconditions hold: a failure classifier upstream of dispatch, a confidence floor on the propose-fix branch, and reviewer attention that is not at capacity. Missing any one transfers cost to reviewers instead of removing it.
Established — multiple independent sources Last reviewed: 2026-06-03
Feedback