Reward Modeling on UltraFeedback core250 (held-out evaluation)
3.543Delta (Δ)TEA
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| TEAN (completions per prompt)=128, Policy Backbone=Qwen/Qwen3-4B-Instruct-2507, Reward Model=Skywork/Skywork-Reward-V2-Llama-3.1-8B2026.05 | 3.543 | — | — | — | 72.8 | 1.2 | 26 | 25.425 | |
| TEAN (completions per prompt)=256, Policy Backbone=Qwen/Qwen3-4B-Instruct-2507, Reward Model=Skywork/Skywork-Reward-V2-Llama-3.1-8B2026.05 | 3.542 | — | — | — | 73.6 | 0 | 26.4 | 26.184 | |
| TEAN (completions per prompt)=64, Policy Backbone=Qwen/Qwen3-4B-Instruct-2507, Reward Model=Skywork/Skywork-Reward-V2-Llama-3.1-8B2026.05 | 3.424 | — | — | — | 73.6 | 0 | 26.4 | 24.509 | |
| TEAN (completions per prompt)=32, Policy Backbone=Qwen/Qwen3-4B-Instruct-2507, Reward Model=Skywork/Skywork-Reward-V2-Llama-3.1-8B2026.05 | 3.204 | — | — | — | 73.2 | 0 | 26.8 | 23.406 | |
| TEAN (completions per prompt)=16, Policy Backbone=Qwen/Qwen3-4B-Instruct-2507, Reward Model=Skywork/Skywork-Reward-V2-Llama-3.1-8B2026.05 | 2.888 | — | — | — | 71.6 | 0 | 28.4 | 22.127 | |
| TEAN (completions per prompt)=8, Policy Backbone=Qwen/Qwen3-4B-Instruct-2507, Reward Model=Skywork/Skywork-Reward-V2-Llama-3.1-8B2026.05 | 2.453 | — | — | — | 66.8 | 0 | 33.2 | 20.588 | |
| TEAN (completions per prompt)=4, Policy Backbone=Qwen/Qwen3-4B-Instruct-2507, Reward Model=Skywork/Skywork-Reward-V2-Llama-3.1-8B2026.05 | 1.929 | — | — | — | 63.2 | 0 | 36.8 | 18.727 | |
| TEAN (completions per prompt)=2, Policy Backbone=Qwen/Qwen3-4B-Instruct-2507, Reward Model=Skywork/Skywork-Reward-V2-Llama-3.1-8B2026.05 | 1.392 | — | — | — | 59.2 | 0 | 40.8 | 16.564 | |
| TEAN (completions per prompt)=1, Policy Backbone=Qwen/Qwen3-4B-Instruct-2507, Reward Model=Skywork/Skywork-Reward-V2-Llama-3.1-8B2026.05 | 0.8 | — | — | — | 53.6 | 0 | 46.4 | 14.001 | |
| TEAN=64, Reward Model=Skywork-Reward-V2-Qwen3-8B2026.05 | 0.292 | — | 6.284 | 0.164 | 60.8 | 0.8 | 38.4 | — | |
| TEAN=1, Reward Model=Skywork-Reward-V2-Qwen3-8B2026.05 | 0.279 | — | -0.735 | 0.146 | 65.6 | 0 | 34.4 | — | |
| TEAN=32, Reward Model=Skywork-Reward-V2-Qwen3-8B2026.05 | 0.249 | — | 5.614 | 0.13 | 63.6 | 0.8 | 35.6 | — | |
| TEAN=128, Reward Model=Skywork-Reward-V2-Qwen3-8B2026.05 | 0.248 | — | 6.801 | 0.109 | 58.8 | 1.6 | 39.6 | — | |
| TEAN=2, Reward Model=Skywork-Reward-V2-Qwen3-8B2026.05 | 0.24 | — | 1.34 | 0.118 | 70 | 0 | 30 | — | |
| TEAN=4, Reward Model=Skywork-Reward-V2-Qwen3-8B2026.05 | 0.212 | — | 2.831 | 0.1 | 70.8 | 0 | 29.2 | — | |
| TEAN=16, Reward Model=Skywork-Reward-V2-Qwen3-8B2026.05 | 0.212 | — | 4.858 | 0.1 | 67.6 | 0 | 32.4 | — | |
| TEAN=256, Reward Model=Skywork-Reward-V2-Qwen3-8B2026.05 | 0.209 | — | 7.303 | 0.005 | 56 | 3.2 | 40.8 | — | |
| TEAN=8, Reward Model=Skywork-Reward-V2-Qwen3-8B2026.05 | 0.196 | — | 3.96 | 0.097 | 69.6 | 0 | 30.4 | — | |
| GRPON=1, Reward Model=Skywork-Reward-V2-Qwen3-8B2026.05 | — | -1.013 | — | — | — | — | — | — | |
| GRPON=2, Reward Model=Skywork-Reward-V2-Qwen3-8B2026.05 | — | 1.1 | — | — | — | — | — | — | |
| GRPON=4, Reward Model=Skywork-Reward-V2-Qwen3-8B2026.05 | — | 2.619 | — | — | — | — | — | — | |
| GRPON=8, Reward Model=Skywork-Reward-V2-Qwen3-8B2026.05 | — | 3.764 | — | — | — | — | — | — | |
| GRPON=16, Reward Model=Skywork-Reward-V2-Qwen3-8B2026.05 | — | 4.646 | — | — | — | — | — | — | |
| GRPON=32, Reward Model=Skywork-Reward-V2-Qwen3-8B2026.05 | — | 5.364 | — | — | — | — | — | — | |
| GRPON=64, Reward Model=Skywork-Reward-V2-Qwen3-8B2026.05 | — | 5.992 | — | — | — | — | — | — | |
| GRPON=128, Reward Model=Skywork-Reward-V2-Qwen3-8B2026.05 | — | 6.553 | — | — | — | — | — | — | |
| GRPON=256, Reward Model=Skywork-Reward-V2-Qwen3-8B2026.05 | — | 7.093 | — | — | — | — | — | — | |
| GRPON (completions per prompt)=1, Policy Backbone=Qwen/Qwen3-4B-Instruct-2507, Reward Model=Skywork/Skywork-Reward-V2-Llama-3.1-8B2026.05 | — | — | — | — | — | — | — | 13.202 | |
| GRPON (completions per prompt)=2, Policy Backbone=Qwen/Qwen3-4B-Instruct-2507, Reward Model=Skywork/Skywork-Reward-V2-Llama-3.1-8B2026.05 | — | — | — | — | — | — | — | 15.173 | |
| GRPON (completions per prompt)=4, Policy Backbone=Qwen/Qwen3-4B-Instruct-2507, Reward Model=Skywork/Skywork-Reward-V2-Llama-3.1-8B2026.05 | — | — | — | — | — | — | — | 16.797 | |
| GRPON (completions per prompt)=8, Policy Backbone=Qwen/Qwen3-4B-Instruct-2507, Reward Model=Skywork/Skywork-Reward-V2-Llama-3.1-8B2026.05 | — | — | — | — | — | — | — | 18.135 | |
| GRPON (completions per prompt)=16, Policy Backbone=Qwen/Qwen3-4B-Instruct-2507, Reward Model=Skywork/Skywork-Reward-V2-Llama-3.1-8B2026.05 | — | — | — | — | — | — | — | 19.239 | |
| GRPON (completions per prompt)=32, Policy Backbone=Qwen/Qwen3-4B-Instruct-2507, Reward Model=Skywork/Skywork-Reward-V2-Llama-3.1-8B2026.05 | — | — | — | — | — | — | — | 20.202 | |
| GRPON (completions per prompt)=64, Policy Backbone=Qwen/Qwen3-4B-Instruct-2507, Reward Model=Skywork/Skywork-Reward-V2-Llama-3.1-8B2026.05 | — | — | — | — | — | — | — | 21.085 | |
| GRPON (completions per prompt)=128, Policy Backbone=Qwen/Qwen3-4B-Instruct-2507, Reward Model=Skywork/Skywork-Reward-V2-Llama-3.1-8B2026.05 | — | — | — | — | — | — | — | 21.882 | |
| GRPON (completions per prompt)=256, Policy Backbone=Qwen/Qwen3-4B-Instruct-2507, Reward Model=Skywork/Skywork-Reward-V2-Llama-3.1-8B2026.05 | — | — | — | — | — | — | — | 22.642 |