Learning to Defer on CIFAR100 (test)
98.9CoverageA-SM
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| A-SMBackbone=WideResNet-28, Expert Accuracy=75%, Expert Knowledge Classes=202023.11 | 98.9 | 24.58 | 4.34 | 24.58 | 24.58 | 24.58 | — | |
| A-SMBackbone=WideResNet-28, Expert Accuracy=94%, Expert Knowledge Classes=202023.11 | 98.16 | 24.54 | 4.63 | 24.54 | 24.54 | 24.54 | — | |
| A-SMBackbone=WideResNet-28, Expert Accuracy=75%, Expert Knowledge Classes=402023.11 | 96.53 | 24.29 | 5.58 | 24.29 | 24.29 | 24.29 | — | |
| A-OvABackbone=WideResNet-28, Expert Accuracy=75%, Expert Knowledge Classes=202023.11 | 93.05 | 25.94 | 4.78 | 25.94 | 25.94 | 25.94 | — | |
| A-SMBackbone=WideResNet-28, Expert Accuracy=75%, Expert Knowledge Classes=602023.11 | 92.63 | 22.48 | 5.58 | 22.48 | 22.48 | 22.48 | — | |
| A-SMBackbone=WideResNet-28, Expert Accuracy=94%, Expert Knowledge Classes=402023.11 | 92.2 | 22.17 | 6.58 | 22.17 | 22.17 | 22.17 | — | |
| A-OvABackbone=WideResNet-28, Expert Accuracy=94%, Expert Knowledge Classes=202023.11 | 90.4 | 25.63 | 4.35 | 25.65 | 25.63 | 25.63 | — | |
| S-OvABackbone=WideResNet-28, Expert Accuracy=75%, Expert Knowledge Classes=202023.11 | 90.06 | 26.53 | 5.57 | 26.83 | 26.53 | 26.53 | — | |
| S-SMBackbone=WideResNet-28, Expert Accuracy=75%, Expert Knowledge Classes=202023.11 | 86.46 | 25.49 | 5.07 | 26.09 | 25.48 | 25.49 | — | |
| S-OvABackbone=WideResNet-28, Expert Accuracy=94%, Expert Knowledge Classes=202023.11 | 85.4 | 27 | 5.15 | 28.42 | 27 | 27 | — | |
| A-SMBackbone=WideResNet-28, Expert Accuracy=94%, Expert Knowledge Classes=602023.11 | 84.72 | 19.3 | 5.96 | 22.91 | 19.3 | 19.3 | — | |
| A-OvABackbone=WideResNet-28, Expert Accuracy=75%, Expert Knowledge Classes=402023.11 | 83.29 | 25.73 | 5.75 | 27.75 | 25.73 | 25.73 | — | |
| TDEFSetup=I, Cost type=error2026.04 | 83.26 | 23.99 | — | — | — | — | — | |
| MILDSetup=I, Cost type=error2026.04 | 81.21 | 22.72 | — | — | — | — | — | |
| A-OvABackbone=WideResNet-28, Expert Accuracy=94%, Expert Knowledge Classes=402023.11 | 80.46 | 23.23 | 6.81 | 27.44 | 23.23 | 23.23 | — | |
| A-OvABackbone=WideResNet-28, Expert Accuracy=75%, Expert Knowledge Classes=602023.11 | 79.76 | 22.81 | 6.1 | 27.19 | 23.08 | 22.81 | — | |
| S-SMBackbone=WideResNet-28, Expert Accuracy=94%, Expert Knowledge Classes=202023.11 | 77.72 | 24.58 | 4.86 | 31.68 | 25 | 24.59 | — | |
| S-OvABackbone=WideResNet-28, Expert Accuracy=75%, Expert Knowledge Classes=402023.11 | 77.69 | 26.89 | 7.14 | 32.3 | 28.33 | 26.9 | — | |
| S-SMBackbone=WideResNet-28, Expert Accuracy=75%, Expert Knowledge Classes=402023.11 | 74.65 | 25.13 | 12.82 | 32.82 | 26.99 | 25.13 | — | |
| A-OvABackbone=WideResNet-28, Expert Accuracy=94%, Expert Knowledge Classes=602023.11 | 70.44 | 19.64 | 7.34 | 31.9 | 24.75 | 20.05 | — | |
| S-OvABackbone=WideResNet-28, Expert Accuracy=75%, Expert Knowledge Classes=602023.11 | 69.08 | 25.29 | 7.47 | 36.67 | 30.44 | 25.7 | — | |
| S-OvABackbone=WideResNet-28, Expert Accuracy=94%, Expert Knowledge Classes=402023.11 | 68.47 | 27.33 | 9.38 | 39.95 | 32.93 | 27.92 | — | |
| S-SMBackbone=WideResNet-28, Expert Accuracy=75%, Expert Knowledge Classes=602023.11 | 60.25 | 24.05 | 18.7 | 42.3 | 36.18 | 29.47 | — | |
| S-OvABackbone=WideResNet-28, Expert Accuracy=94%, Expert Knowledge Classes=602023.11 | 58.18 | 18.44 | 7.7 | 42.52 | 33.67 | 25.74 | — | |
| S-SMBackbone=WideResNet-28, Expert Accuracy=94%, Expert Knowledge Classes=402023.11 | 57.89 | 21.92 | 9.22 | 46.69 | 38.5 | 29.94 | — | |
| TDEFSetup=I, Cost type=error + cost2026.04 | 57.13 | — | — | — | — | — | 81.5 | |
| MILDSetup=II, Cost type=error2026.04 | 55.21 | 28.99 | — | — | — | — | — | |
| TDEFSetup=II, Cost type=error2026.04 | 53.71 | 29.82 | — | — | — | — | — | |
| MILDSetup=I, Cost type=error + cost2026.04 | 50.43 | — | — | — | — | — | 79.28 | |
| TDEFSetup=III, Cost type=error2026.04 | 46.35 | 32.15 | — | — | — | — | — | |
| MILDSetup=II, Cost type=error + cost2026.04 | 45.04 | — | — | — | — | — | 65.06 | |
| TDEFSetup=II, Cost type=error + cost2026.04 | 42.92 | — | — | — | — | — | 66.87 | |
| MILDSetup=III, Cost type=error2026.04 | 42.69 | 31.28 | — | — | — | — | — | |
| S-SMBackbone=WideResNet-28, Expert Accuracy=94%, Expert Knowledge Classes=602023.11 | 40.34 | 18.69 | 11.1 | 59.44 | 51.36 | 42.77 | — | |
| TDEFSetup=III, Cost type=error + cost2026.04 | 37.46 | — | — | — | — | — | 60.51 | |
| MILDSetup=III, Cost type=error + cost2026.04 | 32.23 | — | — | — | — | — | 58.59 |