When one agent fails and the whole multi-agent system breaks with it

What this risk is

The failure or misbehavior of one agent in a pipeline propagates to downstream agents, which amplify rather than contain the error, producing system-level outcomes far more severe than the original failure. The system as a whole fails catastrophically even though each individual agent may be operating within its designed parameters.

—

How it occurs · Mechanisms

Cascading failures in agentic pipelines follow a predictable pattern:

1. Initial failure — One agent produces an incorrect or unexpected output (hallucination, misclassification, reasoning error)
2. Trust propagation — The next agent treats the flawed output as ground truth, building its reasoning on the error
3. Amplification — Each subsequent agent compounds the error, potentially taking actions based on increasingly distorted information
4. Irreversible action — At some point in the pipeline, an agent takes an action in the real world (sends message, executes code, makes transaction) based on accumulated errors
5. Cascade — Real-world consequences trigger further automated responses, spreading the failure

—

Mitigations · Governance

System Design Controls

• Circuit breakers — Automatic pipeline halts when outputs deviate beyond defined bounds
• Idempotency — Design agent actions to be safely repeatable; avoid actions that compound if executed multiple times
• Checkpointing — Validate pipeline state at defined checkpoints before proceeding
• Output bounds validation — Each agent validates that its output falls within expected bounds before passing to the next agent
• Dead letter queues — Failed or anomalous agent outputs go to human review queues rather than downstream agents

Operational Controls

• Blast radius limitation — Restrict the real-world impact any single pipeline execution can have (max transaction size, max emails sent, max files modified)
• Canary deployments — Test pipeline changes on a small subset before full rollout
• Rollback capability — Design all agent-initiated actions to be reversible where possible

—