Preference Calibration on PPE
0.034KuiperProbe
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ProbeModel Architecture=J1 LLAMA 70B2025.12 | 0.034 | 0.039 | |
| MajorityModel Architecture=QWEN 32B2025.12 | 0.038 | 0.065 | |
| ProbeModel Architecture=LLAMA 70B2025.12 | 0.04 | 0.047 | |
| ConsistencyModel Architecture=QWEN 32B2025.12 | 0.048 | 0.083 | |
| MajorityModel Architecture=J1 QWEN 32B2025.12 | 0.05 | 0.069 | |
| ConsistencyModel Architecture=J1 QWEN 32B2025.12 | 0.051 | 0.071 | |
| ProbeModel Architecture=J1 LLAMA 8B2025.12 | 0.051 | 0.066 | |
| ProbeModel Architecture=J1 QWEN 32B2025.12 | 0.056 | 0.072 | |
| ProbeModel Architecture=LLAMA 8B2025.12 | 0.062 | 0.093 | |
| ProbeModel Architecture=QWEN 32B2025.12 | 0.066 | 0.077 | |
| VerbalizedModel Architecture=J1 LLAMA 70B2025.12 | 0.122 | 0.12 | |
| VerbalizedModel Architecture=LLAMA 70B2025.12 | 0.123 | 0.121 | |
| MajorityModel Architecture=LLAMA 8B2025.12 | 0.15 | 0.143 | |
| ConsistencyModel Architecture=LLAMA 8B2025.12 | 0.153 | 0.155 | |
| VerbalizedModel Architecture=LLAMA 8B2025.12 | 0.173 | 0.166 | |
| VerbalizedModel Architecture=J1 LLAMA 8B2025.12 | 0.192 | 0.188 | |
| VerbalizedModel Architecture=J1 QWEN 32B2025.12 | 0.195 | 0.195 | |
| VerbalizedModel Architecture=QWEN 32B2025.12 | 0.196 | 0.194 | |
| ConsistencyModel Architecture=J1 LLAMA 70B2025.12 | 0.207 | 0.2 | |
| MajorityModel Architecture=J1 LLAMA 70B2025.12 | 0.211 | 0.197 | |
| ConsistencyModel Architecture=LLAMA 70B2025.12 | 0.231 | 0.226 | |
| MajorityModel Architecture=LLAMA 70B2025.12 | 0.234 | 0.223 | |
| MajorityModel Architecture=J1 LLAMA 8B2025.12 | 0.323 | 0.31 | |
| ConsistencyModel Architecture=J1 LLAMA 8B2025.12 | 0.324 | 0.311 |