# Reward Modeling on PPE Pref

Source: [Reward Modeling on PPE Pref](<https://www.sota2.com/research/sota/reward-modeling-on-ppe-pref>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-04-10T07:19:22.311Z

Evaluation of model performance on the PPE Pref dataset for the Reward Modeling task, measured by Accuracy.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Reward Modeling |
| Dataset (including any stated split) | PPE Pref |
| Metric | Accuracy |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 15 of 15 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Accuracy](<https://www.sota2.com/research/sota/reward-modeling-on-ppe-pref.md?metric=Accuracy>): Higher is better; unit: PERCENT.

## Recorded results

| Method | Accuracy | Result date | Report paper |
| --- | --- | --- | --- |
| ConsistRM (Qwen3-8B) | 67.7 | Not recorded | 69d883df7d0172140ec777ee |
| DeepSeek-GRM-27B | 67.2 | Not recorded | 69d883df7d0172140ec777ee |
| Qwen3-8B + RFT | 65.4 | Not recorded | 69d883df7d0172140ec777ee |
| ConsistRM (Qwen3-4B) | 65.1 | Not recorded | 69d883df7d0172140ec777ee |
| Qwen3-8B + SFT | 65 | Not recorded | 69d883df7d0172140ec777ee |
| Qwen3-8B + TTRL | 65 | Not recorded | 69d883df7d0172140ec777ee |
| Qwen3-8B + CAAR | 64.8 | Not recorded | 69d883df7d0172140ec777ee |
| Qwen3-4B + RFT | 64.7 | Not recorded | 69d883df7d0172140ec777ee |
| Qwen3-4B + CAAR | 64.1 | Not recorded | 69d883df7d0172140ec777ee |
| Qwen3-8B | 63.8 | Not recorded | 69d883df7d0172140ec777ee |
| Qwen3-4B | 63.3 | Not recorded | 69d883df7d0172140ec777ee |
| Qwen3-4B + SFT | 63.1 | Not recorded | 69d883df7d0172140ec777ee |
| Qwen3-4B + TTRL | 61.8 | Not recorded | 69d883df7d0172140ec777ee |
| J1-Llama-8B | 59.8 | Not recorded | 69d883df7d0172140ec777ee |
| EvalPlanner-Llama-8B | 54.3 | Not recorded | 69d883df7d0172140ec777ee |

### ConsistRM (Qwen3-8B)

Accuracy: 67.7. Result ID: 69d8a46047f34ce924c1187f.

[Reporting paper](<https://www.sota2.com/research/paper/69d883df7d0172140ec777ee.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69d883df7d0172140ec777ee.md>)

Recorded evaluation configuration: {"Base Model":"Qwen3-8B","Training Method":"CAAR + CACR","GT":"False"}. This record does not assert independent reproduction.

### DeepSeek-GRM-27B

Accuracy: 67.2. Result ID: 69d8a46047f34ce924c11873.

[Reporting paper](<https://www.sota2.com/research/paper/69d883df7d0172140ec777ee.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7b951e209131cea89458.md>)

Recorded evaluation configuration: {"Base Model":"DeepSeek-27B","GT":"True"}. This record does not assert independent reproduction.

### Qwen3-8B + RFT

Accuracy: 65.4. Result ID: 69d8a46047f34ce924c1187c.

[Reporting paper](<https://www.sota2.com/research/paper/69d883df7d0172140ec777ee.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73101e209131cea5a395.md>)

[Implementation](<https://github.com/deepseek-ai/DeepSeek-Math>)

Recorded evaluation configuration: {"Base Model":"Qwen3-8B","Training Method":"RFT","GT":"True"}. This record does not assert independent reproduction.

### ConsistRM (Qwen3-4B)

Accuracy: 65.1. Result ID: 69d8a46047f34ce924c11879.

[Reporting paper](<https://www.sota2.com/research/paper/69d883df7d0172140ec777ee.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69d883df7d0172140ec777ee.md>)

Recorded evaluation configuration: {"Base Model":"Qwen3-4B","Training Method":"CAAR + CACR","GT":"False"}. This record does not assert independent reproduction.

### Qwen3-8B + SFT

Accuracy: 65. Result ID: 69d8a46047f34ce924c1187b.

[Reporting paper](<https://www.sota2.com/research/paper/69d883df7d0172140ec777ee.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78a01e209131cea796d7.md>)

Recorded evaluation configuration: {"Base Model":"Qwen3-8B","Training Method":"SFT","GT":"True"}. This record does not assert independent reproduction.

### Qwen3-8B + TTRL

Accuracy: 65. Result ID: 69d8a46047f34ce924c1187d.

[Reporting paper](<https://www.sota2.com/research/paper/69d883df7d0172140ec777ee.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa79ce1e209131cea7f7aa.md>)

Recorded evaluation configuration: {"Base Model":"Qwen3-8B","Training Method":"TTRL","GT":"False"}. This record does not assert independent reproduction.

### Qwen3-8B + CAAR

Accuracy: 64.8. Result ID: 69d8a46047f34ce924c1187e.

[Reporting paper](<https://www.sota2.com/research/paper/69d883df7d0172140ec777ee.md>)

Recorded evaluation configuration: {"Base Model":"Qwen3-8B","Training Method":"CAAR","GT":"False"}. This record does not assert independent reproduction.

### Qwen3-4B + RFT

Accuracy: 64.7. Result ID: 69d8a46047f34ce924c11876.

[Reporting paper](<https://www.sota2.com/research/paper/69d883df7d0172140ec777ee.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73101e209131cea5a395.md>)

[Implementation](<https://github.com/deepseek-ai/DeepSeek-Math>)

Recorded evaluation configuration: {"Base Model":"Qwen3-4B","Training Method":"RFT","GT":"True"}. This record does not assert independent reproduction.

### Qwen3-4B + CAAR

Accuracy: 64.1. Result ID: 69d8a46047f34ce924c11878.

[Reporting paper](<https://www.sota2.com/research/paper/69d883df7d0172140ec777ee.md>)

Recorded evaluation configuration: {"Base Model":"Qwen3-4B","Training Method":"CAAR","GT":"False"}. This record does not assert independent reproduction.

### Qwen3-8B

Accuracy: 63.8. Result ID: 69d8a46047f34ce924c1187a.

[Reporting paper](<https://www.sota2.com/research/paper/69d883df7d0172140ec777ee.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa788e1e209131cea79116.md>)

Recorded evaluation configuration: {"Base Model":"Qwen3-8B"}. This record does not assert independent reproduction.

### Qwen3-4B

Accuracy: 63.3. Result ID: 69d8a46047f34ce924c11874.

[Reporting paper](<https://www.sota2.com/research/paper/69d883df7d0172140ec777ee.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa788e1e209131cea79116.md>)

Recorded evaluation configuration: {"Base Model":"Qwen3-4B"}. This record does not assert independent reproduction.

### Qwen3-4B + SFT

Accuracy: 63.1. Result ID: 69d8a46047f34ce924c11875.

[Reporting paper](<https://www.sota2.com/research/paper/69d883df7d0172140ec777ee.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78a01e209131cea796d7.md>)

Recorded evaluation configuration: {"Base Model":"Qwen3-4B","Training Method":"SFT","GT":"True"}. This record does not assert independent reproduction.

### Qwen3-4B + TTRL

Accuracy: 61.8. Result ID: 69d8a46047f34ce924c11877.

[Reporting paper](<https://www.sota2.com/research/paper/69d883df7d0172140ec777ee.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa79ce1e209131cea7f7aa.md>)

Recorded evaluation configuration: {"Base Model":"Qwen3-4B","Training Method":"TTRL","GT":"False"}. This record does not assert independent reproduction.

### J1-Llama-8B

Accuracy: 59.8. Result ID: 69d8a46047f34ce924c11872.

[Reporting paper](<https://www.sota2.com/research/paper/69d883df7d0172140ec777ee.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7c191e209131cea8c528.md>)

Recorded evaluation configuration: {"Base Model":"Llama-8B","GT":"True"}. This record does not assert independent reproduction.

### EvalPlanner-Llama-8B

Accuracy: 54.3. Result ID: 69d8a46047f34ce924c11871.

[Reporting paper](<https://www.sota2.com/research/paper/69d883df7d0172140ec777ee.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa79ef1e209131cea80190.md>)

Recorded evaluation configuration: {"Base Model":"Llama-8B","GT":"True"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
