Confidence Estimation on APIGen (test)
0.02smECEHRE
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| HREestimator=HRE2026.06 | 0.02 | 82 | 70 | 82 | 78 | |
| NWKRestimator=NWKR2026.06 | 0.02 | 82 | 69 | 82 | 78 | |
| IsotonicModel=Qwen3-30B-A3B-Instruct-25072026.06 | 0.021 | 93.8 | 75.3 | 68.8 | 79.5 | |
| PlattModel=Qwen3-30B-A3B-Instruct-25072026.06 | 0.025 | 93.8 | 75.3 | 66.9 | 78.8 | |
| ACUTE pca10Model=Qwen3-30B-A3B-Instruct-25072026.06 | 0.025 | 94.6 | 83.4 | 81.3 | 86.5 | |
| ACUTE pca20Model=Qwen3-30B-A3B-Instruct-25072026.06 | 0.027 | 94.7 | 84.3 | 81.2 | 86.8 | |
| HREModel=Qwen3-30B-A3B-Instruct-25072026.06 | 0.028 | 94.1 | 79.9 | 71.8 | 82 | |
| ACUTEestimator=acute cosine2026.06 | 0.03 | 87 | 77 | 84 | 83 | |
| ACUTE cosineModel=Qwen3-30B-A3B-Instruct-25072026.06 | 0.031 | 94.6 | 81.7 | 75.7 | 84.1 | |
| BetaModel=Qwen3-30B-A3B-Instruct-25072026.06 | 0.036 | 93.8 | 75.3 | 67.9 | 79.1 | |
| ACUTEestimator=acute pca102026.06 | 0.04 | 90 | 82 | 87 | 87 | |
| NWKRModel=Qwen3-30B-A3B-Instruct-25072026.06 | 0.043 | 93.8 | 76 | 68.8 | 79.7 | |
| ACUTE early actModel=Qwen3-30B-A3B-Instruct-25072026.06 | 0.047 | 94.2 | 82.5 | 78.4 | 85.1 | |
| ACUTEestimator=acute early act2026.06 | 0.05 | 90 | 82 | 87 | 86 | |
| ACUTE mid actModel=Qwen3-30B-A3B-Instruct-25072026.06 | 0.057 | 94.3 | 84.4 | 82.1 | 87 | |
| ACUTEestimator=acute mid act2026.06 | 0.06 | 90 | 84 | 88 | 87 | |
| ACUTEestimator=acute pca202026.06 | 0.06 | 91 | 84 | 88 | 88 | |
| ACUTE late actModel=Qwen3-30B-A3B-Instruct-25072026.06 | 0.063 | 94.3 | 85.1 | 82.2 | 87.2 | |
| ACUTEestimator=acute late act2026.06 | 0.07 | 90 | 84 | 88 | 87 | |
| Raw Confestimator=Raw Conf2026.06 | 0.22 | 28 | 53 | 80 | 53 | |
| Raw ConfModel=Qwen3-30B-A3B-Instruct-25072026.06 | 0.261 | 46.5 | 47.1 | 59.4 | 51 |