Accuracy and Consistency Metrics on MT-Bench (LLM-as-a-Judge)
81.4AccuracyPA-GRPO
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| PA-GRPOModel=Qwen3-32B2026.03 | 81.4 | 91.6 | 74.8 | |
| GRPOModel=Qwen3-32B2026.03 | 81 | 90.6 | 73.7 | |
| PIFModel=Qwen3-32B2026.03 | 79.4 | 86.5 | 72.8 | |
| PA-GRPOModel=Qwen3-8B2026.03 | 78.8 | 86.4 | 72 | |
| CalibraEvalModel=Qwen3-32B2026.03 | 78.8 | 79.1 | 71.2 | |
| BaseModel=Qwen3-32B2026.03 | 78.5 | 83.4 | 70.3 | |
| GRPOModel=Qwen3-8B2026.03 | 78.1 | 82.5 | 69.3 | |
| PA-GRPOModel=Llama-3.1-8B2026.03 | 77.6 | 88 | 71.7 | |
| PriDeModel=Qwen3-32B2026.03 | 77.2 | 86.1 | 72.1 | |
| UniBiasModel=Qwen3-32B2026.03 | 76.5 | 85.6 | 71.1 | |
| PIFModel=Llama-3.1-8B2026.03 | 76.1 | 84.6 | 70.4 | |
| GRPOModel=Llama-3.1-8B2026.03 | 75.7 | 80.6 | 65.4 | |
| PIFModel=Qwen3-8B2026.03 | 75.1 | 83.2 | 68.9 | |
| PriDeModel=Qwen3-8B2026.03 | 72.6 | 66.7 | 52.1 | |
| CalibraEvalModel=Qwen3-8B2026.03 | 72.4 | 62.1 | 50.9 | |
| BaseModel=Qwen3-8B2026.03 | 71.2 | 56.3 | 49.4 | |
| UniBiasModel=Llama-3.1-8B2026.03 | 68.5 | 59.4 | 48.2 | |
| UniBiasModel=Qwen3-8B2026.03 | 67 | 52.7 | 43.4 | |
| PriDeModel=Llama-3.1-8B2026.03 | 65.6 | 46.2 | 38.9 | |
| CalibraEvalModel=Llama-3.1-8B2026.03 | 62.3 | 42.1 | 33.4 | |
| BaseModel=Llama-3.1-8B2026.03 | 59.6 | 25.2 | 22.2 | |
| GeniiBackbone=Qwen2.5-14B-Instruct, Evaluation Setup=Same Model Family2025.10 | 58.42 | — | — | |
| VanillaBackbone=Qwen2.5-14B-Instruct, Evaluation Setup=Same Model Family2025.10 | 56.99 | — | — | |
| GeniiBackbone=Qwen2.5-7B-Instruct, Evaluation Setup=Different Model Families2025.10 | 56.42 | — | — | |
| GeniiBackbone=Gemma2-9B-Instruct, Evaluation Setup=Different Model Families2025.10 | 56.13 | — | — | |
| VanillaBackbone=Qwen2.5-7B-Instruct, Evaluation Setup=Different Model Families2025.10 | 55.44 | — | — | |
| VanillaBackbone=Qwen2.5-7B-Instruct, Evaluation Setup=Same Model Family2025.10 | 55.44 | — | — | |
| Self-ConsistencyBackbone=Qwen2.5-7B-Instruct, Evaluation Setup=Different Model Families2025.10 | 55.12 | — | — | |
| GeniiBackbone=Qwen2.5-7B-Instruct, Evaluation Setup=Same Model Family2025.10 | 54.96 | — | — | |
| Self-ConsistencyBackbone=Gemma2-9B-Instruct, Evaluation Setup=Different Model Families2025.10 | 53 | — | — | |
| VanillaBackbone=Gemma2-9B-Instruct, Evaluation Setup=Different Model Families2025.10 | 52.16 | — | — | |
| Long ReasoningBackbone=Llama3.1-8B-Instruct, Evaluation Setup=Different Model Families2025.10 | 51.27 | — | — | |
| Judgment SFTBackbone=Qwen2.5-7B-Instruct, Evaluation Setup=Different Model Families2025.10 | 50.52 | — | — | |
| RepromptBackbone=Qwen2.5-7B-Instruct, Evaluation Setup=Different Model Families2025.10 | 50.31 | — | — | |
| Judgment SFTBackbone=Gemma2-9B-Instruct, Evaluation Setup=Different Model Families2025.10 | 49.54 | — | — | |
| GeniiBackbone=Qwen2.5-3B-Instruct, Evaluation Setup=Same Model Family2025.10 | 49.51 | — | — | |
| Long ReasoningBackbone=Qwen2.5-7B-Instruct, Evaluation Setup=Different Model Families2025.10 | 48.26 | — | — | |
| Judgment SFTBackbone=Llama3.1-8B-Instruct, Evaluation Setup=Different Model Families2025.10 | 46.26 | — | — | |
| VanillaBackbone=Qwen2.5-3B-Instruct, Evaluation Setup=Same Model Family2025.10 | 45.66 | — | — | |
| RepromptBackbone=Gemma2-9B-Instruct, Evaluation Setup=Different Model Families2025.10 | 44.62 | — | — | |
| GeniiBackbone=Llama3.1-8B-Instruct, Evaluation Setup=Different Model Families2025.10 | 43.4 | — | — | |
| Self-ConsistencyBackbone=Llama3.1-8B-Instruct, Evaluation Setup=Different Model Families2025.10 | 42.4 | — | — | |
| VanillaBackbone=Llama3.1-8B-Instruct, Evaluation Setup=Different Model Families2025.10 | 40.86 | — | — | |
| RepromptBackbone=Llama3.1-8B-Instruct, Evaluation Setup=Different Model Families2025.10 | 31.43 | — | — |