LLM Root-Cause Identification (n=30 per model)
75.9Exact RecallWM-SAR
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| WM-SARLLM Backbone=Gemini-2.5-Flash2026.07 | 75.9 | 0.38 | 0.007 | |
| WM-SARLLM Backbone=GPT-4o-mini2026.07 | 50 | 0.23 | 0.065 | |
| WM-SARLLM Backbone=GPT-4o2026.07 | 43.3 | 0.13 | 0.42 | |
| LocalRepair-2HopLLM Backbone=Gemini-2.5-Flash2026.07 | 37.9 | — | — | |
| LocalRepair-2HopLLM Backbone=GPT-4o2026.07 | 30 | — | — | |
| LocalRepair-2HopLLM Backbone=GPT-4o-mini2026.07 | 26.7 | — | — | |
| TopK-5LLM Backbone=GPT-4o2026.07 | 13.3 | — | — | |
| TopK-5LLM Backbone=Gemini-2.5-Flash2026.07 | 11.1 | — | — | |
| TopK-5LLM Backbone=GPT-4o-mini2026.07 | 10 | — | — | |
| Window-4LLM Backbone=GPT-4o-mini2026.07 | 0 | — | — | |
| Window-4LLM Backbone=GPT-4o2026.07 | 0 | — | — | |
| Window-4LLM Backbone=Gemini-2.5-Flash2026.07 | 0 | — | — | |
| Greedy-PointLLM Backbone=GPT-4o-mini2026.07 | 0 | — | — | |
| Greedy-PointLLM Backbone=GPT-4o2026.07 | 0 | — | — | |
| Greedy-PointLLM Backbone=Gemini-2.5-Flash2026.07 | 0 | — | — |