Error Detection on HotpotQA
81AUROCEnsemble-A
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Ensemble-AModel=GPT 5.12026.02 | 81 | 77 | 69 | 73 | |
| DBA-AModel=Llama 3.3 70B2026.02 | 81 | 85 | 84 | 84 | |
| Ensemble-AModel=GPT 5.12026.02 | 81 | — | — | 73 | |
| DBA-AModel=Llama 3.3 70B2026.02 | 81 | — | — | 84 | |
| |S| Hybridn (sampling budget)=82026.04 | 79.7 | — | — | — | |
| DBA-AModel=GPT 5.12026.02 | 79 | 78 | 64 | 70 | |
| DBA-AModel=GPT 5.12026.02 | 79 | — | — | 70 | |
| Hb Pluginn (sampling budget)=102026.04 | 78.4 | — | — | — | |
| Ensemble-IModel=GPT 5.12026.02 | 78 | 67 | 68 | 68 | |
| DBA-IModel=Llama 3.3 70B2026.02 | 78 | 83 | 80 | 81 | |
| Ensemble-AModel=Llama 3.3 70B2026.02 | 78 | 79 | 90 | 84 | |
| Ensemble-IModel=GPT 5.12026.02 | 78 | — | — | 68 | |
| DBA-IModel=Llama 3.3 70B2026.02 | 78 | — | — | 81 | |
| Ensemble-AModel=Llama 3.3 70B2026.02 | 78 | — | — | 84 | |
| SHADEn (sampling budget)=82026.04 | 77.2 | — | — | — | |
| Hb Hybridn (sampling budget)=82026.04 | 77.1 | — | — | — | |
| NumSetsn (sampling budget)=102026.04 | 76.8 | — | — | — | |
| |S| Hybridn (sampling budget)=102026.04 | 76.5 | — | — | — | |
| DBA-AModel=Qwen3-8B2026.02 | 76 | 91 | 83 | 87 | |
| DBA-AModel=Qwen 8B2026.02 | 76 | — | — | 87 | |
| DBA-IModel=GPT 5.12026.02 | 75 | 68 | 61 | 64 | |
| Ensemble-IModel=Llama 3.3 70B2026.02 | 75 | 78 | 85 | 81 | |
| Ensemble-AModel=Qwen3-8B2026.02 | 75 | 89 | 93 | 91 | |
| DBA-IModel=GPT 5.12026.02 | 75 | — | — | 64 | |
| Ensemble-IModel=Llama 3.3 70B2026.02 | 75 | — | — | 81 | |
| Ensemble-AModel=Qwen 8B2026.02 | 75 | — | — | 91 | |
| NumSetsn (sampling budget)=82026.04 | 74.7 | — | — | — | |
| Hb Pluginn (sampling budget)=82026.04 | 74.6 | — | — | — | |
| SHADEn (sampling budget)=102026.04 | 72.5 | — | — | — | |
| Hb Hybridn (sampling budget)=102026.04 | 72.4 | — | — | — | |
| IC-IDKModel=GPT 5.12026.02 | 71 | 74 | 47 | 58 | |
| IC-IDKModel=Llama 3.3 70B2026.02 | 71 | 78 | 71 | 74 | |
| AYSModel=Qwen3-8B2026.02 | 71 | 91 | 65 | 76 | |
| IC-IDKModel=GPT 5.12026.02 | 71 | — | — | 58 | |
| IC-IDKModel=Llama 3.3 70B2026.02 | 71 | — | — | 74 | |
| AYSModel=Qwen 8B2026.02 | 71 | — | — | 76 | |
| DBA-IModel=Qwen3-8B2026.02 | 70 | 88 | 80 | 84 | |
| Ensemble-IModel=Qwen3-8B2026.02 | 70 | 87 | 93 | 89 | |
| DBA-IModel=Qwen 8B2026.02 | 70 | — | — | 84 | |
| Ensemble-IModel=Qwen 8B2026.02 | 70 | — | — | 89 | |
| SHADEn (sampling budget)=52026.04 | 69.8 | — | — | — | |
| Hb Hybridn (sampling budget)=52026.04 | 69.7 | — | — | — | |
| |S| Hybridn (sampling budget)=52026.04 | 68.4 | — | — | — | |
| AYSModel=GPT 5.12026.02 | 67 | 87 | 36 | 51 | |
| AYSModel=GPT 5.12026.02 | 67 | — | — | 51 | |
| NumSetsn (sampling budget)=52026.04 | 66.7 | — | — | — | |
| Hb Pluginn (sampling budget)=52026.04 | 66.1 | — | — | — | |
| IC-IDKModel=Qwen3-8B2026.02 | 65 | 91 | 47 | 62 | |
| IC-IDKModel=Qwen 8B2026.02 | 65 | — | — | 62 | |
| DSEn (sampling budget)=102026.04 | 64.5 | — | — | — | |
| KLEn (sampling budget)=102026.04 | 63.8 | — | — | — | |
| DSEn (sampling budget)=52026.04 | 62.7 | — | — | — | |
| KLEn (sampling budget)=52026.04 | 62.5 | — | — | — | |
| KLEn (sampling budget)=82026.04 | 62.2 | — | — | — | |
| AYSModel=Llama 3.3 70B2026.02 | 62 | 79 | 40 | 53 | |
| AYSModel=Llama 3.3 70B2026.02 | 62 | — | — | 53 | |
| DSEn (sampling budget)=82026.04 | 57.7 | — | — | — |