Reward Model Evaluation on Reward Bench 2 (test)
0.451RB2 Factuality MAEDistribution-Calibrated Aggregation
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Distribution-Calibrated Aggregationn (sample count)=12, Base Model=gemini-2.5-flash2025.12 | 0.451 | 0.287 | 0.285 | 0.414 | 0.285 | 0.081 | |
| Distribution-Calibrated Aggregationn (sample count)=4, Base Model=gemini-2.5-flash2025.12 | 0.487 | 0.332 | 0.306 | 0.451 | 0.319 | 0.094 | |
| USCn (sample count)=12, Base Model=gemini-2.5-flash2025.12 | 0.573 | 0.423 | 0.415 | 0.53 | 0.405 | 0.221 | |
| USCn (sample count)=4, Base Model=gemini-2.5-flash2025.12 | 0.575 | 0.424 | 0.41 | 0.574 | 0.407 | 0.226 | |
| GSCn (sample count)=12, Base Model=gemini-2.5-flash2025.12 | 0.591 | 0.441 | 0.45 | 0.524 | 0.414 | 0.197 | |
| GSCn (sample count)=4, Base Model=gemini-2.5-flash2025.12 | 0.599 | 0.439 | 0.427 | 0.597 | 0.406 | 0.208 | |
| SCn (sample count)=4, Base Model=gemini-2.5-flash2025.12 | 0.615 | 0.394 | 0.36 | 0.498 | 0.373 | 0.155 | |
| SCn (sample count)=12, Base Model=gemini-2.5-flash2025.12 | 0.647 | 0.403 | 0.384 | 0.552 | 0.402 | 0.158 | |
| CI-SCn (sample count)=12, Base Model=gemini-2.5-flash2025.12 | 0.67 | 0.415 | 0.385 | 0.57 | 0.405 | 0.165 | |
| CI-SCn (sample count)=4, Base Model=gemini-2.5-flash2025.12 | 0.675 | 0.415 | 0.391 | 0.551 | 0.412 | 0.178 | |
| Soft-SCn (sample count)=4, Base Model=gemini-2.5-flash2025.12 | 0.681 | 0.397 | 0.4 | 0.581 | 0.406 | 0.177 | |
| Soft-SCn (sample count)=12, Base Model=gemini-2.5-flash2025.12 | 0.711 | 0.37 | 0.372 | 0.603 | 0.409 | 0.177 |