LLM Root-Cause Identification on Synthetic Testbed (n=150)
50Exact RecallWM-SAR
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| WM-SARLLM Backbone=GPT-4o-mini2026.07 | 50 | 50 | 98.7 | 855 | 8.3 | |
| Full-Graph-LLMLLM Backbone=GPT-4o-mini2026.07 | 24 | 24.7 | 95.3 | 2,157 | 25.7 | |
| LLMRepair-Full-Plan-LLMLLM Backbone=GPT-4o-mini2026.07 | 18 | 18 | 98 | 2,299 | 25.7 | |
| LocalRepair-2Hop-LLMLLM Backbone=GPT-4o-mini2026.07 | 16.7 | 17.3 | 95.3 | 1,596 | 18.7 | |
| TraceScan-Full-LLMLLM Backbone=GPT-4o-mini2026.07 | 16.7 | 18 | 93.3 | 1,284 | 25.7 | |
| TopK-5-LLMLLM Backbone=GPT-4o-mini2026.07 | 10.7 | 12.7 | 89.3 | 609 | 5 | |
| Window-8-LLMLLM Backbone=GPT-4o-mini2026.07 | 8.7 | 10.7 | 88.7 | 735 | 8 | |
| Window-4-LLMLLM Backbone=GPT-4o-mini2026.07 | 6 | 8 | 81.3 | 515 | 4 | |
| TraceScan-w2-LLMLLM Backbone=GPT-4o-mini2026.07 | 2 | 8 | 76 | 397 | 2 | |
| TraceScan-w1-LLMLLM Backbone=GPT-4o-mini2026.07 | 0.7 | 3.3 | 70 | 354 | 1 | |
| Greedy-Point-LLMLLM Backbone=GPT-4o-mini2026.07 | 0.7 | 3.3 | 70 | 363 | 1 |