Incorrectness detection on SQuAD
74.2AUC ScoreSHADE
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SHADEn (sampling budget)=82026.04 | 74.2 | 68.1 | |
| Hb Hybridn (sampling budget)=82026.04 | 74.1 | 68 | |
| NumSetsn (sampling budget)=82026.04 | 73.9 | 67.5 | |
| NumSetsn (sampling budget)=102026.04 | 73.5 | 68.3 | |
| SHADEn (sampling budget)=102026.04 | 73.3 | 67.9 | |
| Hb Hybridn (sampling budget)=102026.04 | 73.2 | 67.7 | |
| Hb Pluginn (sampling budget)=102026.04 | 73.1 | 68.3 | |
| Hb Pluginn (sampling budget)=82026.04 | 72.7 | 67.4 | |
| SHADEn (sampling budget)=52026.04 | 71.1 | 66.6 | |
| Hb Hybridn (sampling budget)=52026.04 | 71 | 66.5 | |
| NumSetsn (sampling budget)=52026.04 | 71 | 63.2 | |
| |S| Hybridn (sampling budget)=82026.04 | 70.1 | 68.2 | |
| Hb Pluginn (sampling budget)=52026.04 | 69.9 | 64.4 | |
| |S| Hybridn (sampling budget)=52026.04 | 69.6 | 68.2 | |
| |S| Hybridn (sampling budget)=102026.04 | 67.7 | 65.4 | |
| KLEn (sampling budget)=82026.04 | 61.8 | 63.5 | |
| DSEn (sampling budget)=52026.04 | 61.1 | 57.1 | |
| KLEn (sampling budget)=102026.04 | 61.1 | 64.4 | |
| DSEn (sampling budget)=102026.04 | 60.3 | 59.1 | |
| KLEn (sampling budget)=52026.04 | 59 | 62.3 | |
| DSEn (sampling budget)=82026.04 | 57.3 | 57.6 |