Model Calibration on ImageNet 1k (val)
0.4509ECEMIR
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| MIRModel=BeiT, Framework=base2023.06 | 0.4509 | 0.5376 | 0.1065 | 1.8039 | |
| MIRModel=BeiT, Framework=PROCAL (ours)2023.06 | 0.536 | 0.5455 | 0.1395 | 1.2645 | |
| PTSModel=ViT, Framework=PROCAL (ours)2023.06 | 0.794 | 0.7124 | 0.26 | 1.751 | |
| ConfModel=BeiT, Framework=PROCAL (ours)2023.06 | 0.8573 | 0.7205 | 0.2866 | 1.5379 | |
| MIRModel=ViT, Framework=PROCAL (ours)2023.06 | 0.8638 | 0.8537 | 0.2182 | 1.6818 | |
| MIRModel=ViT, Framework=base2023.06 | 0.8934 | 0.818 | 0.1893 | 1.7695 | |
| PTSKModel=ViT, Framework=PROCAL (ours)2023.06 | 0.8966 | 0.8139 | 0.2957 | 1.8211 | |
| ConfModel=ViT, Framework=PROCAL (ours)2023.06 | 0.9016 | 0.7554 | 0.2543 | 1.754 | |
| TSModel=ViT, Framework=PROCAL (ours)2023.06 | 0.9047 | 0.788 | 0.2806 | 1.7562 | |
| ETSModel=ViT, Framework=PROCAL (ours)2023.06 | 0.9074 | 0.7732 | 0.2684 | 1.7533 | |
| MIRModel=Mixer, Framework=PROCAL (ours)2023.06 | 0.9272 | 0.936 | 0.143 | 2.2112 | |
| MIRModel=ResNet50, Framework=PROCAL (ours)2023.06 | 0.9533 | 0.8436 | 0.2095 | 1.8776 | |
| PTSModel=BeiT, Framework=PROCAL (ours)2023.06 | 0.9787 | 0.8106 | 0.424 | 1.689 | |
| ETSModel=BeiT, Framework=PROCAL (ours)2023.06 | 0.993 | 0.9023 | 0.4333 | 1.5872 | |
| PTSModel=ResNet50, Framework=PROCAL (ours)2023.06 | 1.0138 | 1.0278 | 0.2529 | 2.0445 | |
| MIRModel=ResNet50, Framework=base2023.06 | 1.0281 | 0.9643 | 0.2062 | 1.9751 | |
| PTSKModel=BeiT, Framework=PROCAL (ours)2023.06 | 1.0288 | 0.8582 | 0.5022 | 1.6168 | |
| PTSModel=ViT, Framework=base2023.06 | 1.0609 | 1.0257 | 0.3937 | 2.0929 | |
| ETSModel=ViT, Framework=base2023.06 | 1.108 | 1.1791 | 0.2745 | 1.9273 | |
| MIRModel=Mixer, Framework=base2023.06 | 1.1128 | 1.219 | 0.2628 | 3.3912 | |
| ConfModel=ViT, Framework=base2023.06 | 1.1815 | 1.1839 | 0.3489 | 1.9984 | |
| PTSModel=BeiT, Framework=base2023.06 | 1.2685 | 1.2744 | 0.4858 | 1.9782 | |
| TSModel=BeiT, Framework=PROCAL (ours)2023.06 | 1.3277 | 1.2493 | 0.7046 | 1.9264 | |
| ETSModel=Mixer, Framework=PROCAL (ours)2023.06 | 1.3586 | 1.221 | 0.2986 | 2.201 | |
| TSModel=ViT, Framework=base2023.06 | 1.4905 | 1.4465 | 0.5069 | 2.1462 | |
| PTSKModel=ResNet50, Framework=PROCAL (ours)2023.06 | 1.6015 | 1.5015 | 0.498 | 2.5003 | |
| PTSKModel=ViT, Framework=base2023.06 | 1.6493 | 1.5874 | 0.6035 | 2.5798 | |
| TSModel=Mixer, Framework=PROCAL (ours)2023.06 | 1.6499 | 1.4455 | 0.4255 | 2.4809 | |
| ETSModel=ResNet50, Framework=PROCAL (ours)2023.06 | 1.6548 | 1.6627 | 0.5676 | 2.4579 | |
| TSModel=ResNet50, Framework=PROCAL (ours)2023.06 | 1.7964 | 1.73 | 0.6881 | 2.5721 | |
| PTSKModel=BeiT, Framework=base2023.06 | 1.8861 | 1.915 | 0.7934 | 2.7093 | |
| ETSModel=Mixer, Framework=base2023.06 | 1.9381 | 2.1859 | 0.3204 | 4.1034 | |
| PTSKModel=Mixer, Framework=PROCAL (ours)2023.06 | 1.9466 | 1.7905 | 0.6131 | 2.7503 | |
| ETSModel=BeiT, Framework=base2023.06 | 2.1318 | 2.1862 | 1.2155 | 2.9592 | |
| PTSModel=ResNet50, Framework=base2023.06 | 2.2196 | 2.2098 | 0.6319 | 4.0331 | |
| PTSModel=Mixer, Framework=PROCAL (ours)2023.06 | 2.2208 | 2.1298 | 0.694 | 3.0623 | |
| ETSModel=ResNet50, Framework=base2023.06 | 2.762 | 3.6581 | 0.6624 | 3.575 | |
| ConfModel=ResNet50, Framework=PROCAL (ours)2023.06 | 2.7752 | 2.6344 | 1.3005 | 3.4224 | |
| ConfModel=Mixer, Framework=PROCAL (ours)2023.06 | 2.8498 | 2.7162 | 1.0944 | 3.6485 | |
| TSModel=BeiT, Framework=base2023.06 | 2.9894 | 3.1132 | 0.7388 | 3.5366 | |
| ConfModel=BeiT, Framework=base2023.06 | 3.6137 | 3.5464 | 1.5801 | 4.2348 | |
| PTSKModel=ResNet50, Framework=base2023.06 | 4.41 | 4.3761 | 1.8375 | 5.4413 | |
| HBModel=ViT, Framework=base2023.06 | 4.692 | 7.1771 | 2.4332 | 7.3239 | |
| HBModel=BeiT, Framework=base2023.06 | 4.8765 | 6.1728 | 1.8383 | 7.2174 | |
| TSModel=ResNet50, Framework=base2023.06 | 5.1181 | 5.0864 | 2.5503 | 5.464 | |
| TSModel=Mixer, Framework=base2023.06 | 5.1937 | 5.0234 | 2.0189 | 5.8958 | |
| HBModel=BeiT, Framework=PROCAL (ours)2023.06 | 5.5631 | 5.9747 | 4.1239 | 6.2886 | |
| PTSModel=Mixer, Framework=base2023.06 | 5.7741 | 5.8027 | 2.8498 | 9.3215 | |
| HBModel=ResNet50, Framework=PROCAL (ours)2023.06 | 6.1812 | 7.538 | 4.3484 | 7.7372 | |
| PTSKModel=Mixer, Framework=base2023.06 | 6.661 | 6.6173 | 3.1011 | 8.3148 | |
| HBModel=Mixer, Framework=PROCAL (ours)2023.06 | 6.7207 | 7.4825 | 4.3997 | 7.8672 | |
| HBModel=ViT, Framework=PROCAL (ours)2023.06 | 6.8776 | 7.5824 | 4.8925 | 7.7552 | |
| HBModel=ResNet50, Framework=base2023.06 | 7.6311 | 9.3289 | 2.6377 | 10.1849 | |
| ConfModel=ResNet50, Framework=base2023.06 | 8.7246 | 8.6852 | 4.6122 | 8.9113 | |
| HBModel=Mixer, Framework=base2023.06 | 9.1774 | 9.7965 | 3.5964 | 12.924 | |
| ConfModel=Mixer, Framework=base2023.06 | 10.9366 | 10.9337 | 5.1519 | 11.0164 |