Confidence Estimation on SCITLDR (dev)
0.08smECEHRE
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| HREestimator=HRE2026.06 | 0.08 | 0.67 | 0.71 | 0.92 | 0.77 | |
| NWKRestimator=NWKR2026.06 | 0.08 | 0.67 | 0.71 | 0.92 | 0.77 | |
| ACUTEestimator=acute early act2026.06 | 0.08 | 0.69 | 0.72 | 0.92 | 0.78 | |
| ACUTEestimator=acute mid act2026.06 | 0.08 | 0.7 | 0.72 | 0.92 | 0.78 | |
| ACUTEestimator=acute late act2026.06 | 0.08 | 0.69 | 0.72 | 0.92 | 0.77 | |
| ACUTEestimator=acute cosine2026.06 | 0.08 | 0.68 | 0.71 | 0.92 | 0.77 | |
| PlattModel=Qwen3-30B-A3B-Instruct-25072026.06 | 0.082 | 0.643 | 0.763 | 0.94 | 0.781 | |
| IsotonicModel=Qwen3-30B-A3B-Instruct-25072026.06 | 0.082 | 0.637 | 0.763 | 0.94 | 0.778 | |
| NWKRModel=Qwen3-30B-A3B-Instruct-25072026.06 | 0.083 | 0.641 | 0.763 | 0.94 | 0.78 | |
| BetaModel=Qwen3-30B-A3B-Instruct-25072026.06 | 0.083 | 0.641 | 0.761 | 0.94 | 0.779 | |
| ACUTE late actModel=Qwen3-30B-A3B-Instruct-25072026.06 | 0.087 | 0.654 | 0.764 | 0.94 | 0.784 | |
| HREModel=Qwen3-30B-A3B-Instruct-25072026.06 | 0.088 | 0.637 | 0.763 | 0.94 | 0.779 | |
| ACUTE mid actModel=Qwen3-30B-A3B-Instruct-25072026.06 | 0.088 | 0.653 | 0.763 | 0.94 | 0.784 | |
| ACUTE cosineModel=Qwen3-30B-A3B-Instruct-25072026.06 | 0.089 | 0.634 | 0.76 | 0.94 | 0.776 | |
| ACUTE early actModel=Qwen3-30B-A3B-Instruct-25072026.06 | 0.089 | 0.652 | 0.763 | 0.94 | 0.784 | |
| ACUTEestimator=acute pca102026.06 | 0.09 | 0.7 | 0.72 | 0.92 | 0.78 | |
| ACUTEestimator=acute pca202026.06 | 0.09 | 0.7 | 0.72 | 0.92 | 0.78 | |
| ACUTE pca10Model=Qwen3-30B-A3B-Instruct-25072026.06 | 0.092 | 0.637 | 0.762 | 0.94 | 0.778 | |
| ACUTE pca20Model=Qwen3-30B-A3B-Instruct-25072026.06 | 0.092 | 0.653 | 0.765 | 0.94 | 0.785 | |
| Raw ConfModel=Qwen3-30B-A3B-Instruct-25072026.06 | 0.146 | 0.413 | 0.76 | 0.939 | 0.701 | |
| Raw Confestimator=Raw Conf2026.06 | 0.15 | 0.36 | 0.71 | 0.92 | 0.66 |