Calibration on CelebA ImageResNet (test)
0.39ECE (Oracle Estimate)Platt
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| PlattDistribution Setting (Base, Post)=(PN, UU), Backbone=ImageResNet2026.05 | 0.39 | 3.52 | 24 | 0.69 | 3.78 | 25 | 94.41 | |
| PlattDistribution Setting (Base, Post)=(PN, PU), Backbone=ImageResNet2026.05 | 0.43 | 3.44 | 25 | 0.8 | 4.96 | 29 | 94.42 | |
| PlattDistribution Setting (Base, Post)=(PU, PU), Backbone=ImageResNet2026.05 | 0.81 | 6.15 | 31 | 0.93 | 6.28 | 31 | 93.22 | |
| PlattDistribution Setting (Base, Post)=(UU, UU), Backbone=ImageResNet2026.05 | 0.82 | 5.33 | 35 | 1.06 | 5.89 | 36 | 93.5 | |
| TempDistribution Setting (Base, Post)=(PN, UU), Backbone=ImageResNet2026.05 | 0.87 | 3.87 | 26 | 1.07 | 4.29 | 27 | 94.38 | |
| TempDistribution Setting (Base, Post)=(PN, PU), Backbone=ImageResNet2026.05 | 0.9 | 3.75 | 28 | 1.1 | 4.9 | 32 | 94.38 | |
| ERMDistribution Setting (Base, Post)=(PN, PN), Backbone=ImageResNet2026.05 | 1.01 | 3.4 | 33 | 1.01 | 3.4 | 33 | 94.38 | |
| WLMCDistribution Setting (Base, Post)=(PN, PN), Backbone=ImageResNet2026.05 | 1.01 | 3.4 | 33 | 1.01 | 3.4 | 33 | 94.38 | |
| ERMDistribution Setting (Base, Post)=(PN, PU), Backbone=ImageResNet2026.05 | 1.01 | 3.4 | 33 | 1.16 | 4.94 | 40 | 94.38 | |
| WLMCDistribution Setting (Base, Post)=(PN, PU), Backbone=ImageResNet2026.05 | 1.01 | 3.4 | 33 | 1.16 | 4.94 | 40 | 94.38 | |
| ERMDistribution Setting (Base, Post)=(PN, UU), Backbone=ImageResNet2026.05 | 1.01 | 3.4 | 33 | 1.12 | 3.74 | 36 | 94.38 | |
| WLMCDistribution Setting (Base, Post)=(PN, UU), Backbone=ImageResNet2026.05 | 1.01 | 3.4 | 33 | 1.12 | 3.74 | 36 | 94.38 | |
| TempDistribution Setting (Base, Post)=(PU, PU), Backbone=ImageResNet2026.05 | 1.05 | 5.48 | 37 | 1.14 | 5.84 | 39 | 93.23 | |
| TempDistribution Setting (Base, Post)=(UU, UU), Backbone=ImageResNet2026.05 | 1.11 | 5.83 | 35 | 1.21 | 6.33 | 36 | 93.5 | |
| WLMCDistribution Setting (Base, Post)=(PU, PU), Backbone=ImageResNet2026.05 | 1.22 | 5.23 | 40 | 1.35 | 6.3 | 42 | 93.23 | |
| WLMCDistribution Setting (Base, Post)=(UU, UU), Backbone=ImageResNet2026.05 | 1.29 | 4.82 | 39 | 1.35 | 5.53 | 38 | 93.5 | |
| ERMDistribution Setting (Base, Post)=(PU, PU), Backbone=ImageResNet2026.05 | 1.33 | 5.21 | 47 | 1.47 | 6.38 | 49 | 93.23 | |
| ERMDistribution Setting (Base, Post)=(UU, UU), Backbone=ImageResNet2026.05 | 1.46 | 4.97 | 56 | 1.51 | 5.29 | 50 | 93.5 | |
| PlattDistribution Setting (Base, Post)=(PN, PN), Backbone=ImageResNet2026.05 | 1.48 | 3.46 | 60 | 1.48 | 3.46 | 60 | 94.42 | |
| TempDistribution Setting (Base, Post)=(PN, PN), Backbone=ImageResNet2026.05 | 1.54 | 4.8 | 43 | 1.54 | 4.8 | 43 | 94.38 |