When a multi-agent AI system fails, developers face a daunting question: which agent was responsible, and at what step did things go wrong? Sifting through lengthy interaction logs to find the root cause is like looking for a needle in a haystack.[reference:25]
Now, researchers from Penn State University and Duke University—in collaboration with Google DeepMind, the University of Washington, Meta, Nanyang Technological University, and Oregon State University—have introduced a new research area called "Automated Failure Attribution" to solve exactly this problem.[reference:26][reference:27] Their work, "Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systems," has been accepted as a Spotlight presentation at ICML 2025.[reference:28]
The Who&When Benchmark
The team built the first benchmark dataset for this task, named Who&When. It contains failure logs from 127 diverse LLM multi-agent systems, covering complex tasks from benchmarks like GAIA and AssistantBench.[reference:29][reference:30] Each failure log is annotated with fine-grained human labels identifying which agent caused the failure, at which step, and why.[reference:31][reference:32]
The dataset includes two subsets: an algorithmically generated set with 191 unique agents across 126 logs, and a hand-crafted set with 5 unique agents across 58 logs.[reference:33]
Three Automated Attribution Methods
The researchers proposed and evaluated three automated methods for failure attribution[reference:34]:
- All-at-Once: A single-pass LLM analysis of the entire log
- Step-by-Step: Sequential review of each step in the conversation
- Binary Search: A divide-and-conquer approach that searches logs efficiently
Experiments using models like GPT-4o showed the best agent-level accuracy reached only 53.5%, while step-level accuracy was a modest 14.2%—worse than random in some cases.[reference:35] Even advanced models like OpenAI o1 and DeepSeek R1 struggled, particularly with longer contexts.[reference:36] Hybrid methods improved results but increased costs.[reference:37]
Why Attribution Is So Hard
The research identifies two core challenges[reference:38]:
- Distinguishing symptoms from root causes in multi-agent error propagation—a failure at step 10 might actually be caused by a mistake at step 3
- Tracing information dependencies beyond temporal order—agents don't just act in sequence; they share information, and errors can propagate across the system in complex ways
The team introduced Information Dependency Graphs (IDGs) to trace causal relationships beyond temporal sequences, providing a new framework for understanding how errors spread through multi-agent systems.[reference:39]
Implications for Production AI
The findings highlight a critical gap in current AI infrastructure. Multi-agent systems are increasingly deployed in production, yet we lack reliable tools to debug them when they fail.[reference:40] The study shows that even the most advanced LLMs cannot reliably perform failure attribution today—a problem that will only grow as multi-agent systems become more complex and autonomous.
All code and data from the study have been fully open-sourced, including the dataset on Hugging Face and the code on GitHub, paving the way for further research and practical tooling.[reference:41][reference:42]
For engineering teams building agentic systems, this research represents an essential first step toward making multi-agent AI reliably debuggable—and ultimately, more trustworthy in production.