Uncertainty Calibration on JudgeBench
0.037KuiperProbe
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ProbeModel Backbone=J1 QWEN 32B2025.12 | 0.037 | 0.048 | |
| ProbeModel Backbone=QWEN 32B2025.12 | 0.055 | 0.059 | |
| ProbeModel Backbone=LLAMA 70B2025.12 | 0.062 | 0.077 | |
| ProbeModel Backbone=LLAMA 8B2025.12 | 0.072 | 0.119 | |
| ProbeModel Backbone=J1 LLAMA 70B2025.12 | 0.075 | 0.074 | |
| MajorityModel Backbone=LLAMA 8B2025.12 | 0.076 | 0.171 | |
| MajorityModel Backbone=J1 QWEN 32B2025.12 | 0.076 | 0.105 | |
| ConsistencyModel Backbone=LLAMA 8B2025.12 | 0.077 | 0.193 | |
| ConsistencyModel Backbone=J1 QWEN 32B2025.12 | 0.087 | 0.124 | |
| VerbalizedModel Backbone=QWEN 32B2025.12 | 0.103 | 0.107 | |
| VerbalizedModel Backbone=J1 QWEN 32B2025.12 | 0.114 | 0.113 | |
| ProbeModel Backbone=J1 LLAMA 8B2025.12 | 0.122 | 0.135 | |
| VerbalizedModel Backbone=LLAMA 70B2025.12 | 0.152 | 0.145 | |
| VerbalizedModel Backbone=J1 LLAMA 70B2025.12 | 0.156 | 0.155 | |
| MajorityModel Backbone=QWEN 32B2025.12 | 0.159 | 0.198 | |
| ConsistencyModel Backbone=J1 LLAMA 70B2025.12 | 0.188 | 0.204 | |
| MajorityModel Backbone=J1 LLAMA 70B2025.12 | 0.2 | 0.182 | |
| VerbalizedModel Backbone=LLAMA 8B2025.12 | 0.205 | 0.188 | |
| ConsistencyModel Backbone=QWEN 32B2025.12 | 0.205 | 0.276 | |
| ConsistencyModel Backbone=LLAMA 70B2025.12 | 0.231 | 0.243 | |
| MajorityModel Backbone=LLAMA 70B2025.12 | 0.238 | 0.224 | |
| MajorityModel Backbone=J1 LLAMA 8B2025.12 | 0.271 | 0.259 | |
| VerbalizedModel Backbone=J1 LLAMA 8B2025.12 | 0.274 | 0.263 | |
| ConsistencyModel Backbone=J1 LLAMA 8B2025.12 | 0.282 | 0.259 |