LLM-as-a-Judge on PreferenceBench (Accuracy, Consistency, Consistent Accuracy)
90.2AccuracyPA-GRPO
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| PA-GRPOModel=Qwen3-32B2026.03 | 90.2 | 92.1 | 86.2 | |
| PIFModel=Qwen3-32B2026.03 | 89.3 | 90.6 | 85.5 | |
| GRPOModel=Qwen3-32B2026.03 | 89.1 | 90.3 | 83.3 | |
| PA-GRPOModel=Qwen3-8B2026.03 | 88.9 | 89.6 | 83.7 | |
| UniBiasModel=Qwen3-32B2026.03 | 87.2 | 91.1 | 84.1 | |
| PA-GRPOModel=Llama-3.1-8B2026.03 | 86.2 | 87.2 | 79.8 | |
| PriDeModel=Qwen3-32B2026.03 | 85.1 | 86.7 | 81.2 | |
| BaseModel=Qwen3-32B2026.03 | 84.9 | 85.1 | 78.7 | |
| CalibraEvalModel=Qwen3-32B2026.03 | 83.7 | 87.2 | 79.9 | |
| GRPOModel=Qwen3-8B2026.03 | 82.9 | 77.3 | 76.2 | |
| BaseModel=Qwen3-8B2026.03 | 82.3 | 70.2 | 67.4 | |
| GRPOModel=Llama-3.1-8B2026.03 | 82.2 | 85.1 | 76.3 | |
| PriDeModel=Llama-3.1-8B2026.03 | 81.6 | 69.1 | 66.2 | |
| PriDeModel=Qwen3-8B2026.03 | 81.5 | 72.6 | 69.7 | |
| UniBiasModel=Llama-3.1-8B2026.03 | 79.7 | 65.6 | 62.5 | |
| PIFModel=Qwen3-8B2026.03 | 77.7 | 64.2 | 58.9 | |
| CalibraEvalModel=Qwen3-8B2026.03 | 76.5 | 72.8 | 70.1 | |
| UniBiasModel=Qwen3-8B2026.03 | 75.8 | 55.4 | 53.5 | |
| CalibraEvalModel=Llama-3.1-8B2026.03 | 70.4 | 44.4 | 42.7 | |
| BaseModel=Llama-3.1-8B2026.03 | 60.8 | 22.6 | 22.1 | |
| PIFModel=Llama-3.1-8B2026.03 | 54.3 | 37.1 | 25.7 |