Learning to Defer on CIFAR-10H (test)
100CoverageClassifier only
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Classifier onlyJ=52026.04 | 100 | — | — | — | — | — | 88.9 | 0.109 | 88.9 | |
| Classifier onlyJ=102026.04 | 100 | — | — | — | — | — | 88.9 | 0.111 | 88.9 | |
| Classifier onlyJ=202026.04 | 100 | — | — | — | — | — | 88.8 | 0.112 | 88.8 | |
| A-SMBackbone=28-layer WideResNet, Training Strategy=Train linear layer on CIFAR-10H2023.11 | 94.2 | 3.65 | 0.94 | 3.65 | 3.65 | 3.65 | — | — | — | |
| A-OvABackbone=28-layer WideResNet, Training Strategy=Train linear layer on CIFAR-10H2023.11 | 92.75 | 4 | 0.95 | 4 | 4 | 4 | — | — | — | |
| DecoupledJ=202026.04 | 53 | — | — | — | — | — | 96.1 | 0.039 | 88.9 | |
| DecoupledJ=52026.04 | 51.7 | — | — | — | — | — | 96 | 0.039 | 89.2 | |
| DecoupledJ=102026.04 | 51.7 | — | — | — | — | — | 95.8 | 0.042 | 89.2 | |
| S-SMBackbone=28-layer WideResNet, Training Strategy=Train linear layer on CIFAR-10H2023.11 | 41.28 | 3.82 | 6.64 | 50.25 | 41 | 31.47 | — | — | — | |
| PiCCEJ=102026.04 | 40 | — | — | — | — | — | 95.7 | 0.043 | 89 | |
| PiCCEJ=202026.04 | 39.9 | — | — | — | — | — | 95.3 | 0.047 | 89 | |
| PiCCEJ=52026.04 | 39 | — | — | — | — | — | 95.4 | 0.045 | 88.9 | |
| OvAJ=52026.04 | 37.5 | — | — | — | — | — | 95.9 | 0.039 | 88.9 | |
| S-OvABackbone=28-layer WideResNet, Training Strategy=Train linear layer on CIFAR-10H2023.11 | 37.08 | 4.3 | 2.46 | 52.75 | 43.31 | 33.23 | — | — | — | |
| A-SMJ=52026.04 | 34.1 | — | — | — | — | — | 95.9 | 0.04 | 89 | |
| OvAJ=202026.04 | 33.5 | — | — | — | — | — | 95.6 | 0.043 | 89 | |
| OvAJ=102026.04 | 28.5 | — | — | — | — | — | 95.4 | 0.046 | 79.9 | |
| Add. CEJ=52026.04 | 24.8 | — | — | — | — | — | 95.3 | 0.045 | 88.9 | |
| Add. CEJ=102026.04 | 24.8 | — | — | — | — | — | 95.2 | 0.047 | 88.8 | |
| Add. CEJ=202026.04 | 18.3 | — | — | — | — | — | 95.1 | 0.049 | 74.2 | |
| A-SMJ=102026.04 | 17 | — | — | — | — | — | 95.2 | 0.047 | 74.7 | |
| A-SMJ=202026.04 | 0.2 | — | — | — | — | — | 94.9 | 0.051 | 47 | |
| Mao25J=52026.04 | 0 | — | — | — | — | — | 95 | 0.047 | 11.6 | |
| Mao25J=102026.04 | 0 | — | — | — | — | — | 94.8 | 0.052 | 7.7 | |
| Mao25J=202026.04 | 0 | — | — | — | — | — | 94.8 | 0.051 | 4.5 |