Reward Modeling Evaluation on Reward Bench Factuality 2
56.6Pairwise AccuracyDistribution-Calibrated Aggregation
Evaluation Results
| Method | Links | |
|---|---|---|
| Distribution-Calibrated Aggregationn=12, Judge LLM=gemini-2.5-flash2025.12 | 56.6 | |
| Distribution-Calibrated Aggregationn=4, Judge LLM=gemini-2.5-flash2025.12 | 53.6 | |
| USCn=12, Judge LLM=gemini-2.5-flash2025.12 | 47.5 | |
| USCn=4, Judge LLM=gemini-2.5-flash2025.12 | 47.2 | |
| GSCn=12, Judge LLM=gemini-2.5-flash2025.12 | 46.1 | |
| SCn=4, Judge LLM=gemini-2.5-flash2025.12 | 45 | |
| GSCn=4, Judge LLM=gemini-2.5-flash2025.12 | 44.5 | |
| SCn=12, Judge LLM=gemini-2.5-flash2025.12 | 42.4 | |
| CI-SCn=12, Judge LLM=gemini-2.5-flash2025.12 | 41.1 | |
| Soft-SCn=4, Judge LLM=gemini-2.5-flash2025.12 | 41 | |
| CI-SCn=4, Judge LLM=gemini-2.5-flash2025.12 | 40.9 | |
| Soft-SCn=12, Judge LLM=gemini-2.5-flash2025.12 | 39.9 | |
| Pointwise RatingModel Family=QWEN3, Model Size=8B2026.02 | 34.1 | |
| Listwise Ranking*Model Family=GEMMA3, Model Size=4B2026.02 | 33.2 | |
| FLIPModel Family=QWEN3, Model Size=4B2026.02 | 31.3 | |
| FLIPModel Family=QWEN3, Model Size=8B2026.02 | 31.1 | |
| Pointwise RatingModel Family=QWEN3, Model Size=4B2026.02 | 30.5 | |
| FLIPModel Family=GEMMA3, Model Size=270M2026.02 | 29.8 | |
| FLIPModel Family=QWEN3, Model Size=1.7B2026.02 | 29.6 | |
| Pairwise Ranking*Model Family=MISTRAL, Model Size=7B2026.02 | 28.5 | |
| FLIPModel Family=QWEN3, Model Size=0.6B2026.02 | 28.4 | |
| FLIPModel Family=OLMO2, Model Size=7B2026.02 | 27.8 | |
| Pairwise Ranking*Model Family=OLMO2, Model Size=7B2026.02 | 27.3 | |
| FLIPModel Family=LLAMA3, Model Size=8B2026.02 | 27.3 | |
| Listwise Ranking*Model Family=MISTRAL, Model Size=7B2026.02 | 27.2 | |
| FLIPModel Family=MISTRAL, Model Size=7B2026.02 | 27.1 | |
| Pairwise Ranking*Model Family=QWEN3, Model Size=4B2026.02 | 26.5 | |
| FLIPModel Family=GEMMA3, Model Size=4B2026.02 | 26.5 | |
| FLIPModel Family=LLAMA3, Model Size=3B2026.02 | 26.4 | |
| FLIPModel Family=LLAMA3, Model Size=1B2026.02 | 26.1 | |
| Pairwise Ranking*Model Family=QWEN3, Model Size=1.7B2026.02 | 26.1 | |
| FLIPModel Family=OLMO2, Model Size=1B2026.02 | 25.8 | |
| Pairwise Ranking*Model Family=GEMMA3, Model Size=4B2026.02 | 25.6 | |
| Listwise Ranking*Model Family=LLAMA3, Model Size=8B2026.02 | 25.1 | |
| FLIPModel Family=GEMMA3, Model Size=1B2026.02 | 24.9 | |
| Pairwise Ranking*Model Family=QWEN3, Model Size=8B2026.02 | 24.5 | |
| Pairwise Ranking*Model Family=LLAMA3, Model Size=8B2026.02 | 22.2 | |
| Pairwise Ranking*Model Family=GEMMA3, Model Size=1B2026.02 | 21.7 | |
| Listwise Ranking*Model Family=OLMO2, Model Size=7B2026.02 | 21.3 | |
| Listwise Ranking*Model Family=GEMMA3, Model Size=1B2026.02 | 20.9 | |
| Listwise Ranking*Model Family=LLAMA3, Model Size=3B2026.02 | 20.5 | |
| Pointwise RatingModel Family=QWEN3, Model Size=1.7B2026.02 | 20.2 | |
| Listwise Ranking*Model Family=QWEN3, Model Size=1.7B2026.02 | 19.8 | |
| Listwise Ranking*Model Family=OLMO2, Model Size=1B2026.02 | 18.3 | |
| Pairwise Ranking*Model Family=LLAMA3, Model Size=3B2026.02 | 16.9 | |
| Pointwise RatingModel Family=GEMMA3, Model Size=270M2026.02 | 16.8 | |
| Listwise Ranking*Model Family=QWEN3, Model Size=0.6B2026.02 | 16.6 | |
| Listwise Ranking*Model Family=LLAMA3, Model Size=1B2026.02 | 16 | |
| Pairwise Ranking*Model Family=QWEN3, Model Size=0.6B2026.02 | 15.8 | |
| Pointwise RatingModel Family=LLAMA3, Model Size=1B2026.02 | 15.6 | |
| Pointwise RatingModel Family=LLAMA3, Model Size=8B2026.02 | 15.6 | |
| Pointwise RatingModel Family=OLMO2, Model Size=1B2026.02 | 15.5 | |
| Listwise Ranking*Model Family=GEMMA3, Model Size=270M2026.02 | 15.3 | |
| Pointwise RatingModel Family=GEMMA3, Model Size=4B2026.02 | 14.6 | |
| Pointwise RatingModel Family=QWEN3, Model Size=0.6B2026.02 | 13.5 | |
| Pointwise RatingModel Family=LLAMA3, Model Size=3B2026.02 | 13.3 | |
| Pairwise Ranking*Model Family=LLAMA3, Model Size=1B2026.02 | 12 | |
| Pairwise Ranking*Model Family=GEMMA3, Model Size=270M2026.02 | 11.3 | |
| Pointwise RatingModel Family=GEMMA3, Model Size=1B2026.02 | 10.2 | |
| Listwise Ranking*Model Family=QWEN3, Model Size=8B2026.02 | 9.8 | |
| Listwise Ranking*Model Family=QWEN3, Model Size=4B2026.02 | 8.9 | |
| Pointwise RatingModel Family=OLMO2, Model Size=7B2026.02 | 7.8 | |
| Pointwise RatingModel Family=MISTRAL, Model Size=7B2026.02 | 5.5 | |
| Pairwise Ranking*Model Family=OLMO2, Model Size=1B2026.02 | 1.2 |