Reward Modeling on RewardBench Precise IF 2
57.5AccuracyGemini-2.5-Flash
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Gemini-2.5-Flash2026.02 | 57.5 | — | |
| Rubric-ARM-voting@5voting=52026.02 | 46.2 | — | |
| API (Rubric+Judge)Rubric Model=GPT-4.1-Mini, Judge Model=Gemini-2.5-Flash Lite2026.02 | 42.5 | — | |
| Rubric-ARM2026.02 | 41.9 | — | |
| Rubric-RM-voting@5voting=52026.02 | 40 | — | |
| Claude-3.5-Sonnet2026.02 | 38.8 | — | |
| RM-R1-32B (DeepSeek-Dist)Backbone=DeepSeek-Dist, Size=32B2026.02 | 36.9 | — | |
| RRM-32BSize=32B2026.02 | 34.4 | — | |
| Rubric-RM2026.02 | 34.4 | — | |
| RM-R1-32B (Qwen-2.5-Inst)Backbone=Qwen-2.5-Inst, Size=32B2026.02 | 33.1 | — | |
| RM-R1-14B (DeepSeek-Dist)Backbone=DeepSeek-Dist, Size=14B2026.02 | 30.6 | — | |
| Listwise Ranking*Model Family=MISTRAL, Model Size=7B2026.02 | 29.2 | — | |
| Pairwise Ranking*Model Family=GEMMA3, Model Size=4B2026.02 | 28.9 | — | |
| Pairwise Ranking*Model Family=MISTRAL, Model Size=7B2026.02 | 28 | — | |
| Pairwise Ranking*Model Family=OLMO2, Model Size=7B2026.02 | 26 | — | |
| FLIPModel Family=QWEN3, Model Size=4B2026.02 | 25 | — | |
| Pairwise Ranking*Model Family=GEMMA3, Model Size=1B2026.02 | 25 | — | |
| FLIPModel Family=QWEN3, Model Size=0.6B2026.02 | 24.6 | — | |
| FLIPModel Family=QWEN3, Model Size=8B2026.02 | 24.4 | — | |
| FLIPModel Family=LLAMA3, Model Size=3B2026.02 | 24.2 | — | |
| RM-R1-14B (Qwen-2.5-Inst)Backbone=Qwen-2.5-Inst, Size=14B2026.02 | 23.8 | — | |
| FLIPModel Family=GEMMA3, Model Size=4B2026.02 | 23.8 | — | |
| FLIPModel Family=QWEN3, Model Size=1.7B2026.02 | 23.6 | — | |
| Listwise Ranking*Model Family=LLAMA3, Model Size=8B2026.02 | 23.5 | — | |
| FLIPModel Family=OLMO2, Model Size=1B2026.02 | 23.2 | — | |
| FLIPModel Family=LLAMA3, Model Size=8B2026.02 | 23.1 | — | |
| FLIPModel Family=LLAMA3, Model Size=1B2026.02 | 23 | — | |
| FLIPModel Family=MISTRAL, Model Size=7B2026.02 | 23 | — | |
| FLIPModel Family=GEMMA3, Model Size=1B2026.02 | 22.2 | — | |
| Qwen-3-8B (Rubric+Judge)Backbone=Qwen-3-8B2026.02 | 21.9 | — | |
| Listwise Ranking*Model Family=OLMO2, Model Size=7B2026.02 | 21.9 | — | |
| FLIPModel Family=GEMMA3, Model Size=270M2026.02 | 21.9 | — | |
| Listwise Ranking*Model Family=GEMMA3, Model Size=1B2026.02 | 21.8 | — | |
| FLIPModel Family=OLMO2, Model Size=7B2026.02 | 21.5 | — | |
| Listwise Ranking*Model Family=OLMO2, Model Size=1B2026.02 | 21.2 | — | |
| Pairwise Ranking*Model Family=LLAMA3, Model Size=8B2026.02 | 20.9 | — | |
| RM-R1-7B (Qwen-2.5-Inst)Backbone=Qwen-2.5-Inst, Size=7B2026.02 | 20.6 | — | |
| Listwise Ranking*Model Family=LLAMA3, Model Size=3B2026.02 | 20.2 | — | |
| Listwise Ranking*Model Family=GEMMA3, Model Size=4B2026.02 | 20.1 | — | |
| Pointwise RatingModel Family=LLAMA3, Model Size=1B2026.02 | 19 | — | |
| Pointwise RatingModel Family=QWEN3, Model Size=8B2026.02 | 17.6 | — | |
| Listwise Ranking*Model Family=QWEN3, Model Size=0.6B2026.02 | 16.1 | — | |
| Listwise Ranking*Model Family=GEMMA3, Model Size=270M2026.02 | 16 | — | |
| Listwise Ranking*Model Family=LLAMA3, Model Size=1B2026.02 | 15.2 | — | |
| Pairwise Ranking*Model Family=QWEN3, Model Size=1.7B2026.02 | 15.2 | — | |
| Pointwise RatingModel Family=GEMMA3, Model Size=270M2026.02 | 15.2 | — | |
| Pairwise Ranking*Model Family=LLAMA3, Model Size=3B2026.02 | 14.2 | — | |
| Listwise Ranking*Model Family=QWEN3, Model Size=1.7B2026.02 | 14.1 | — | |
| Pairwise Ranking*Model Family=QWEN3, Model Size=4B2026.02 | 13.9 | — | |
| Pairwise Ranking*Model Family=GEMMA3, Model Size=270M2026.02 | 13.9 | — | |
| Pointwise RatingModel Family=GEMMA3, Model Size=1B2026.02 | 13.9 | — | |
| RM-R1-7B (DeepSeek-Dist)Backbone=DeepSeek-Dist, Size=7B2026.02 | 13.8 | — | |
| Pairwise Ranking*Model Family=QWEN3, Model Size=8B2026.02 | 13.6 | — | |
| API (direct Judge)Judge Model=Gemini-2.5-Flash Lite2026.02 | 13.2 | — | |
| Pointwise RatingModel Family=QWEN3, Model Size=1.7B2026.02 | 13.1 | — | |
| Pointwise RatingModel Family=OLMO2, Model Size=1B2026.02 | 13 | — | |
| Pointwise RatingModel Family=QWEN3, Model Size=4B2026.02 | 12.8 | — | |
| Pointwise RatingModel Family=LLAMA3, Model Size=3B2026.02 | 12.2 | — | |
| Pairwise Ranking*Model Family=LLAMA3, Model Size=1B2026.02 | 11.6 | — | |
| Pointwise RatingModel Family=QWEN3, Model Size=0.6B2026.02 | 11.4 | — | |
| Pairwise Ranking*Model Family=QWEN3, Model Size=0.6B2026.02 | 10.8 | — | |
| RRM-7BSize=7B2026.02 | 10 | — | |
| Pointwise RatingModel Family=LLAMA3, Model Size=8B2026.02 | 9.9 | — | |
| JudgeLRM-7BSize=7B2026.02 | 9.4 | — | |
| Listwise Ranking*Model Family=QWEN3, Model Size=8B2026.02 | 9.2 | — | |
| Pointwise RatingModel Family=GEMMA3, Model Size=4B2026.02 | 7.9 | — | |
| Pointwise RatingModel Family=OLMO2, Model Size=7B2026.02 | 7.1 | — | |
| Pointwise RatingModel Family=MISTRAL, Model Size=7B2026.02 | 6.1 | — | |
| Listwise Ranking*Model Family=QWEN3, Model Size=4B2026.02 | 4.9 | — | |
| Pairwise Ranking*Model Family=OLMO2, Model Size=1B2026.02 | 2.2 | — | |
| CI-SCn=4, Judge LLM=gemini-2.5-flash2025.12 | — | 52.8 | |
| CI-SCn=12, Judge LLM=gemini-2.5-flash2025.12 | — | 51.5 | |
| Distribution-Calibrated Aggregationn=4, Judge LLM=gemini-2.5-flash2025.12 | — | 57.2 | |
| Distribution-Calibrated Aggregationn=12, Judge LLM=gemini-2.5-flash2025.12 | — | 60.5 | |
| GSCn=4, Judge LLM=gemini-2.5-flash2025.12 | — | 47.4 | |
| GSCn=12, Judge LLM=gemini-2.5-flash2025.12 | — | 52.7 | |
| SCn=4, Judge LLM=gemini-2.5-flash2025.12 | — | 55.6 | |
| SCn=12, Judge LLM=gemini-2.5-flash2025.12 | — | 53 | |
| Soft-SCn=4, Judge LLM=gemini-2.5-flash2025.12 | — | 50.7 | |
| Soft-SCn=12, Judge LLM=gemini-2.5-flash2025.12 | — | 49 | |
| USCn=4, Judge LLM=gemini-2.5-flash2025.12 | — | 49.5 | |
| USCn=12, Judge LLM=gemini-2.5-flash2025.12 | — | 52.2 |