Calibration on CIFAR-10H
0.84ECEP+L (Recalibrated)
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| P+L (Recalibrated)Backbone=ResNet-164, Fitting Datapoints=5000, Recalibration Datapoints=50002021.09 | 0.84 | 0.14 | 0.08 | |
| P+L (Recalibrated)Backbone=ResNet-110, Fitting Datapoints=5000, Recalibration Datapoints=50002021.09 | 0.85 | 0.19 | 0.12 | |
| P+L (Recalibrated)Backbone=DenseNet-BC, Fitting Datapoints=5000, Recalibration Datapoints=50002021.09 | 0.95 | 0.1 | 0.07 | |
| P+L (Recalibrated)Backbone=PreResNet-164, Fitting Datapoints=5000, Recalibration Datapoints=50002021.09 | 1.06 | 0.12 | 0.08 | |
| P+LBackbone=ResNet-164, Fitting Datapoints=102021.09 | 1.25 | 0.15 | 0.1 | |
| P+LCalibration Datapoints=10, Backbone=ResNet-164, Prediction Source=Classifier + Human2021.09 | 1.25 | 0.15 | 0.1 | |
| P+LBackbone=DenseNet-BC, Fitting Datapoints=50002021.09 | 1.27 | 0.11 | 0.08 | |
| P+LCalibration Datapoints=5000, Backbone=DenseNet-BC, Prediction Source=Classifier + Human2021.09 | 1.27 | 0.11 | 0.08 | |
| P+LBackbone=ResNet-110, Fitting Datapoints=102021.09 | 1.3 | 0.24 | 0.15 | |
| P+LCalibration Datapoints=10, Backbone=ResNet-110, Prediction Source=Classifier + Human2021.09 | 1.3 | 0.24 | 0.15 | |
| P+LBackbone=DenseNet-BC, Fitting Datapoints=102021.09 | 1.34 | 0.12 | 0.09 | |
| P+LCalibration Datapoints=10, Backbone=DenseNet-BC, Prediction Source=Classifier + Human2021.09 | 1.34 | 0.12 | 0.09 | |
| P+LBackbone=ResNet-164, Fitting Datapoints=50002021.09 | 1.39 | 0.13 | 0.1 | |
| P+LCalibration Datapoints=5000, Backbone=ResNet-164, Prediction Source=Classifier + Human2021.09 | 1.39 | 0.13 | 0.1 | |
| P+LBackbone=PreResNet-164, Fitting Datapoints=102021.09 | 1.4 | 0.13 | 0.11 | |
| P+LCalibration Datapoints=10, Backbone=PreResNet-164, Prediction Source=Classifier + Human2021.09 | 1.4 | 0.13 | 0.11 | |
| P+LBackbone=PreResNet-164, Fitting Datapoints=50002021.09 | 1.43 | 0.13 | 0.11 | |
| P+LCalibration Datapoints=5000, Backbone=PreResNet-164, Prediction Source=Classifier + Human2021.09 | 1.43 | 0.13 | 0.1 | |
| P+LBackbone=DenseNet-BC, Fitting Datapoints=None2021.09 | 1.53 | 0.11 | 0.1 | |
| P+LCalibration Datapoints=0, Backbone=DenseNet-BC, Prediction Source=Classifier + Human2021.09 | 1.53 | 0.11 | 0.1 | |
| P+LBackbone=ResNet-164, Fitting Datapoints=None2021.09 | 1.63 | 0.15 | 0.11 | |
| P+LCalibration Datapoints=0, Backbone=ResNet-164, Prediction Source=Classifier + Human2021.09 | 1.63 | 0.15 | 0.11 | |
| DenseNet-BCCalibration=Temperature Scaling, Calibration Datapoints=50002021.09 | 1.73 | 0.24 | 0.14 | |
| DenseNet-BCCalibration Datapoints=5000, Architecture=DenseNet-BC, Prediction Source=Classifier Only2021.09 | 1.73 | 0.24 | 0.14 | |
| P+LBackbone=ResNet-110, Fitting Datapoints=50002021.09 | 1.76 | 0.19 | 0.11 | |
| DenseNet-BCCalibration=Temperature Scaling, Calibration Datapoints=102021.09 | 1.76 | 0.24 | 0.14 | |
| DenseNet-BCCalibration Datapoints=10, Architecture=DenseNet-BC, Prediction Source=Classifier Only2021.09 | 1.76 | 0.24 | 0.14 | |
| P+LCalibration Datapoints=5000, Backbone=ResNet-110, Prediction Source=Classifier + Human2021.09 | 1.76 | 0.19 | 0.14 | |
| P+LBackbone=PreResNet-164, Fitting Datapoints=None2021.09 | 1.87 | 0.13 | 0.13 | |
| P+LCalibration Datapoints=0, Backbone=PreResNet-164, Prediction Source=Classifier + Human2021.09 | 1.87 | 0.13 | 0.13 | |
| ResNet-164Calibration=Temperature Scaling, Calibration Datapoints=50002021.09 | 1.89 | 0.31 | 0.2 | |
| ResNet-164Calibration Datapoints=5000, Architecture=ResNet-164, Prediction Source=Classifier Only2021.09 | 1.89 | 0.31 | 0.2 | |
| ResNet-164Calibration=Temperature Scaling, Calibration Datapoints=102021.09 | 1.95 | 0.31 | 0.2 | |
| ResNet-164Calibration Datapoints=10, Architecture=ResNet-164, Prediction Source=Classifier Only2021.09 | 1.95 | 0.31 | 0.2 | |
| P+LBackbone=ResNet-110, Fitting Datapoints=None2021.09 | 2.08 | 0.23 | 0.16 | |
| P+LCalibration Datapoints=0, Backbone=ResNet-110, Prediction Source=Classifier + Human2021.09 | 2.08 | 0.23 | 0.16 | |
| DenseNet-BCCalibration=None2021.09 | 2.18 | 0.23 | 0.17 | |
| DenseNet-BCCalibration Datapoints=0, Architecture=DenseNet-BC, Prediction Source=Classifier Only2021.09 | 2.18 | 0.23 | 0.17 | |
| PreResNet-164Calibration=Temperature Scaling, Calibration Datapoints=50002021.09 | 2.27 | 0.28 | 0.19 | |
| PreResNet-164Calibration Datapoints=5000, Architecture=PreResNet-164, Prediction Source=Classifier Only2021.09 | 2.27 | 0.28 | 0.19 | |
| PreResNet-164Calibration=Temperature Scaling, Calibration Datapoints=102021.09 | 2.31 | 0.28 | 0.19 | |
| PreResNet-164Calibration Datapoints=10, Architecture=PreResNet-164, Prediction Source=Classifier Only2021.09 | 2.31 | 0.28 | 0.19 | |
| ResNet-164Calibration=None2021.09 | 2.98 | 0.39 | 0.24 | |
| ResNet-164Calibration Datapoints=0, Architecture=ResNet-164, Prediction Source=Classifier Only2021.09 | 2.98 | 0.39 | 0.24 | |
| ResNet-110Calibration=Temperature Scaling, Calibration Datapoints=50002021.09 | 2.99 | 0.58 | 0.35 | |
| ResNet-110Calibration Datapoints=5000, Architecture=ResNet-110, Prediction Source=Classifier Only2021.09 | 2.99 | 0.58 | 0.35 | |
| ResNet-110Calibration=Temperature Scaling, Calibration Datapoints=102021.09 | 3.03 | 0.58 | 0.35 | |
| PreResNet-164Calibration=None2021.09 | 3.03 | 0.29 | 0.23 | |
| PreResNet-164Calibration Datapoints=0, Architecture=PreResNet-164, Prediction Source=Classifier Only2021.09 | 3.03 | 0.29 | 0.23 | |
| ResNet-110Calibration Datapoints=10, Architecture=ResNet-110, Prediction Source=Classifier Only2021.09 | 3.03 | 0.58 | 0.35 | |
| ResNet-110Calibration=None2021.09 | 5.23 | 0.81 | 0.4 | |
| ResNet-110Calibration Datapoints=0, Architecture=ResNet-110, Prediction Source=Classifier Only2021.09 | 5.23 | 0.81 | 0.4 |