Learning to Defer on CIFAR-10 (test)
71.31Deferral Ratio (Expert 1)TDEF
Evaluation Results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| TDEFSetup=I, Cost type=error, Backbone=ResNet-16, Batch size=1024, Learning rate=1e-3, Optimizer=Adam, Epochs=2002026.04 | 71.31 | 0.052 | — | 19.18 | 9.51 | — | — | |
| MILDSetup=I, Cost type=error, Backbone=ResNet-16, Batch size=1024, Learning rate=1e-3, Optimizer=Adam, Epochs=2002026.04 | 70.01 | 0.0403 | — | 19.99 | 10 | — | — | |
| MILDSetup=I, Cost type=error + cost, Backbone=ResNet-16, Batch size=1024, Learning rate=1e-3, Optimizer=Adam, Epochs=2002026.04 | 67.02 | — | 0.5779 | 20.04 | 12.94 | — | — | |
| TDEFSetup=I, Cost type=error + cost, Backbone=ResNet-16, Batch size=1024, Learning rate=1e-3, Optimizer=Adam, Epochs=2002026.04 | 63.68 | — | 0.595 | 23.61 | 12.71 | — | — | |
| TDEFSetup=II, Cost type=error, Backbone=ResNet-16, Batch size=1024, Learning rate=1e-3, Optimizer=Adam, Epochs=2002026.04 | 51.54 | 0.0924 | — | 19.53 | 18.76 | 10.17 | — | |
| MILDSetup=II, Cost type=error, Backbone=ResNet-16, Batch size=1024, Learning rate=1e-3, Optimizer=Adam, Epochs=2002026.04 | 51.5 | 0.0847 | — | 19.41 | 18.99 | 10.1 | — | |
| TDEFSetup=II, Cost type=error + cost, Backbone=ResNet-16, Batch size=1024, Learning rate=1e-3, Optimizer=Adam, Epochs=2002026.04 | 48.11 | — | 0.4421 | 22.81 | 18.31 | 10.77 | — | |
| MILDSetup=II, Cost type=error + cost, Backbone=ResNet-16, Batch size=1024, Learning rate=1e-3, Optimizer=Adam, Epochs=2002026.04 | 44.87 | — | 0.424 | 22.65 | 20.5 | 11.98 | — | |
| MILDSetup=III, Cost type=error, Backbone=ResNet-16, Batch size=1024, Learning rate=1e-3, Optimizer=Adam, Epochs=2002026.04 | 40.4 | 0.0903 | — | 20.13 | 19.94 | 10.73 | 8.8 | |
| TDEFSetup=III, Cost type=error, Backbone=ResNet-16, Batch size=1024, Learning rate=1e-3, Optimizer=Adam, Epochs=2002026.04 | 38.94 | 0.1062 | — | 20.93 | 20.9 | 9.87 | 9.36 | |
| MILDSetup=III, Cost type=error + cost, Backbone=ResNet-16, Batch size=1024, Learning rate=1e-3, Optimizer=Adam, Epochs=2002026.04 | 37.55 | — | 0.3512 | 20.26 | 20.48 | 10.09 | 11.62 | |
| TDEFSetup=III, Cost type=error + cost, Backbone=ResNet-16, Batch size=1024, Learning rate=1e-3, Optimizer=Adam, Epochs=2002026.04 | 37.53 | — | 0.3684 | 22.33 | 18.14 | 11.75 | 10.25 |