Learning to Defer on ImageNet varying-accuracy synthetic expert (val)
41.18ErrorPiCCE-CE
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| PiCCE-CELoss Formulation=PiCCE, Expert Pattern=Varying-Acc Dog Expert, #Exp=12, Backbone=ResNet-182026.02 | 41.18 | 90.01 | |
| PiCCE-CELoss Formulation=PiCCE, Expert Pattern=Varying-Acc Dog Expert, #Exp=16, Backbone=ResNet-182026.02 | 41.21 | 90.07 | |
| PiCCE-CELoss Formulation=PiCCE, Expert Pattern=Varying-Acc Dog Expert, #Exp=8, Backbone=ResNet-182026.02 | 41.28 | 89.93 | |
| PiCCE-CELoss Formulation=PiCCE, Expert Pattern=Varying-Acc Dog Expert, #Exp=20, Backbone=ResNet-182026.02 | 41.39 | 90.1 | |
| PiCCE-CELoss Formulation=PiCCE, Expert Pattern=Varying-Acc Dog Expert, #Exp=4, Backbone=ResNet-182026.02 | 41.87 | 90.02 | |
| PiCCE-OvALoss Formulation=PiCCE, Expert Pattern=Varying-Acc Dog Expert, #Exp=4, Backbone=ResNet-182026.02 | 42.12 | 89.4 | |
| PiCCE-OvALoss Formulation=PiCCE, Expert Pattern=Varying-Acc Dog Expert, #Exp=20, Backbone=ResNet-182026.02 | 42.13 | 89.24 | |
| PiCCE-OvALoss Formulation=PiCCE, Expert Pattern=Varying-Acc Dog Expert, #Exp=16, Backbone=ResNet-182026.02 | 42.14 | 89.32 | |
| PiCCE-OvALoss Formulation=PiCCE, Expert Pattern=Varying-Acc Dog Expert, #Exp=8, Backbone=ResNet-182026.02 | 42.35 | 89.51 | |
| PiCCE-OvALoss Formulation=PiCCE, Expert Pattern=Varying-Acc Dog Expert, #Exp=12, Backbone=ResNet-182026.02 | 42.51 | 89.26 | |
| CELoss Formulation=Vanilla, Expert Pattern=Varying-Acc Dog Expert, #Exp=4, Backbone=ResNet-182026.02 | 42.58 | 89.21 | |
| CELoss Formulation=Vanilla, Expert Pattern=Varying-Acc Dog Expert, #Exp=8, Backbone=ResNet-182026.02 | 44.27 | 88.82 | |
| OvALoss Formulation=Vanilla, Expert Pattern=Varying-Acc Dog Expert, #Exp=4, Backbone=ResNet-182026.02 | 45.03 | 88.24 | |
| CELoss Formulation=Vanilla, Expert Pattern=Varying-Acc Dog Expert, #Exp=12, Backbone=ResNet-182026.02 | 46.28 | 88.27 | |
| CELoss Formulation=Vanilla, Expert Pattern=Varying-Acc Dog Expert, #Exp=16, Backbone=ResNet-182026.02 | 47.02 | 88.71 | |
| CELoss Formulation=Vanilla, Expert Pattern=Varying-Acc Dog Expert, #Exp=20, Backbone=ResNet-182026.02 | 47.98 | 87.81 | |
| OvALoss Formulation=Vanilla, Expert Pattern=Varying-Acc Dog Expert, #Exp=8, Backbone=ResNet-182026.02 | 52.38 | 86.71 | |
| OvALoss Formulation=Vanilla, Expert Pattern=Varying-Acc Dog Expert, #Exp=12, Backbone=ResNet-182026.02 | 58.13 | 85.16 | |
| OvALoss Formulation=Vanilla, Expert Pattern=Varying-Acc Dog Expert, #Exp=16, Backbone=ResNet-182026.02 | 61.02 | 84.54 | |
| OvALoss Formulation=Vanilla, Expert Pattern=Varying-Acc Dog Expert, #Exp=20, Backbone=ResNet-182026.02 | 65.27 | 83.81 |