Probability Calibration on ChaosNLI combined SNLI+MNLI K=3
2.15ECEIR-Soft
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| IR-SoftBackbone=DeBERTa-v32026.03 | 2.15 | 3.96 | 7.2 | 0.554 | 0.942 | |
| Dirichlet-SoftBackbone=RoBERTa-Large2026.03 | 2.57 | 1.71 | 2.03 | 0.51 | 0.839 | |
| IR-SoftBackbone=RoBERTa-Large2026.03 | 2.65 | 2.59 | 6.24 | 0.549 | 0.934 | |
| LS-TSBackbone=DeBERTa-v3, Temperature (T)=6.152026.03 | 2.65 | 2.99 | 6.32 | 0.529 | 0.881 | |
| SoftPlattBackbone=RoBERTa-Large2026.03 | 2.66 | 2.32 | 2.43 | 0.509 | 0.839 | |
| MCTS S=1Backbone=RoBERTa-Large, Temperature (T)=3.492026.03 | 2.82 | 2.35 | 4.6 | 0.519 | 0.862 | |
| SoftPlattBackbone=DeBERTa-v32026.03 | 3.17 | 2.92 | 3.18 | 0.511 | 0.841 | |
| Dirichlet-SoftBackbone=DeBERTa-v32026.03 | 3.19 | 2.71 | 2.95 | 0.509 | 0.836 | |
| MCTS S=1Backbone=DeBERTa-v3, Temperature (T)=5.452026.03 | 3.2 | 2.98 | 6.3 | 0.529 | 0.877 | |
| SLTSBackbone=RoBERTa-Large, Temperature (T)=3.412026.03 | 3.22 | 2.59 | 4.62 | 0.52 | 0.862 | |
| SLTSBackbone=DeBERTa-v3, Temperature (T)=5.282026.03 | 3.45 | 3.36 | 6.53 | 0.529 | 0.877 | |
| VSBackbone=RoBERTa-Large2026.03 | 3.46 | 3.07 | 5.13 | 0.518 | 0.857 | |
| Oracle TSBackbone=RoBERTa-Large, Temperature (T)=3.342026.03 | 3.48 | 2.83 | 4.64 | 0.52 | 0.862 | |
| Oracle TSBackbone=DeBERTa-v3, Temperature (T)=5.212026.03 | 3.66 | 3.54 | 6.66 | 0.529 | 0.876 | |
| VSBackbone=DeBERTa-v32026.03 | 3.91 | 3.35 | 6.46 | 0.525 | 0.869 | |
| LS-TSBackbone=RoBERTa-Large, Temperature (T)=4.402026.03 | 4.16 | 3.72 | 5.61 | 0.522 | 0.873 | |
| TSBackbone=RoBERTa-Large, Temperature (T)=2.422026.03 | 10.55 | 10.38 | 7.3 | 0.537 | 0.886 | |
| PlattBackbone=RoBERTa-Large2026.03 | 11.16 | 10.92 | 7.28 | 0.53 | 0.875 | |
| ATSBackbone=RoBERTa-Large, Temperature (T)=2.282026.03 | 11.36 | 11.29 | 7.73 | 0.538 | 0.883 | |
| Dirichlet-HardBackbone=RoBERTa-Large2026.03 | 11.55 | 11.4 | 7.64 | 0.535 | 0.889 | |
| TSBackbone=DeBERTa-v3, Temperature (T)=3.812026.03 | 11.63 | 11.57 | 8.48 | 0.549 | 0.9 | |
| PlattBackbone=DeBERTa-v32026.03 | 12.43 | 12.36 | 8.43 | 0.539 | 0.896 | |
| Dirichlet-HardBackbone=DeBERTa-v32026.03 | 12.69 | 12.63 | 8.84 | 0.539 | 0.895 | |
| ATSBackbone=DeBERTa-v3, Temperature (T)=3.572026.03 | 13.09 | 13.07 | 9.06 | 0.552 | 0.9 | |
| UncalibratedBackbone=RoBERTa-Large2026.03 | 27.79 | 27.77 | 18.61 | 0.65 | 1.384 | |
| UncalibratedBackbone=DeBERTa-v32026.03 | 35.97 | 35.92 | 24.02 | 0.743 | 2.148 |