Summarization on TL;DR (test)
82.5Win RateGRPO
Evaluation results
| Method | Links | |
|---|---|---|
| GRPOModel Family=Qwen 2.5, Size=1.5B2026.02 | 82.5 | |
| GRPO + Ref. ResetModel Family=Qwen 2.5, Size=1.5B2026.02 | 82 | |
| RLOO + R2MBase Model=Pythia-2.8B2026.01 | 81.6 | |
| GRPO + R2MBase Model=Pythia-2.8B2026.01 | 81 | |
| ReMax+XRLHFBackbone=pythia-2.8B, Evaluator=GPT-42025.12 | 80.4 | |
| PPO+XRLHFBackbone=pythia-2.8B, Evaluator=GPT-42025.12 | 79.2 | |
| ReMax+XRLHFBackbone=Llama-2-7B, Evaluator=GPT-42025.12 | 78.1 | |
| GRPO + Ref. ResetModel Family=Pythia, Size=1B2026.02 | 78.1 | |
| ReMax+XRLHFBackbone=Llama-2-7B, Evaluator=Deberta-v3-large-v22025.12 | 77.4 | |
| PPO+XRLHFBackbone=pythia-2.8B, Evaluator=Deberta-v3-large-v22025.12 | 76.8 | |
| GRPO + Ref. ResetModel Family=Pythia, Size=2.8B2026.02 | 76.4 | |
| ReMax+XRLHFBackbone=pythia-2.8B, Evaluator=Deberta-v3-large-v22025.12 | 75.6 | |
| RLOOBase Model=Pythia-2.8B2026.01 | 75.3 | |
| GRPOBase Model=Pythia-2.8B2026.01 | 75.2 | |
| PPO+XRLHFBackbone=Llama-2-7B, Evaluator=Deberta-v3-large-v22025.12 | 75.2 | |
| ReMaxBase Model=Pythia-2.8B2026.01 | 75.1 | |
| REINFORCE++Base Model=Pythia-2.8B2026.01 | 74.3 | |
| TR-DPOModel Family=Qwen 2.5, Size=1.5B2026.02 | 73.4 | |
| PPO+XRLHFBackbone=Llama-2-7B, Evaluator=GPT-42025.12 | 72.5 | |
| DPOModel Family=Qwen 2.5, Size=1.5B2026.02 | 71.6 | |
| ReMaxBackbone=pythia-2.8B, Evaluator=GPT-42025.12 | 71.2 | |
| PPOBackbone=pythia-2.8B, Evaluator=GPT-42025.12 | 70.4 | |
| GRPO + Ref. ResetModel Family=Qwen 2.5, Size=0.5B2026.02 | 70.2 | |
| ReMaxBackbone=Llama-2-7B, Evaluator=Deberta-v3-large-v22025.12 | 69.6 | |
| GRPOModel Family=Pythia, Size=2.8B2026.02 | 68.5 | |
| PPOBackbone=Llama-2-7B, Evaluator=GPT-42025.12 | 68.4 | |
| DPOModel Family=Pythia, Size=2.8B2026.02 | 68 | |
| ReMaxBackbone=Llama-2-7B, Evaluator=GPT-42025.12 | 67.9 | |
| RLOO + Pretrained RMBase Model=Pythia-2.8B, Reward Model=Pre-trained2026.01 | 67.3 | |
| GRPO + Iterative RMHeadBase Model=Pythia-2.8B, Reward Model=Iterative RMHead2026.01 | 67 | |
| RLOO + Iterative RMHeadBase Model=Pythia-2.8B, Reward Model=Iterative RMHead2026.01 | 66.9 | |
| TR-DPOModel Family=Pythia, Size=2.8B2026.02 | 66.6 | |
| GRPO + Pretrained RMBase Model=Pythia-2.8B, Reward Model=Pre-trained2026.01 | 66.3 | |
| PPOBackbone=pythia-2.8B, Evaluator=Deberta-v3-large-v22025.12 | 65.8 | |
| PPOBackbone=Llama-2-7B, Evaluator=Deberta-v3-large-v22025.12 | 63.6 | |
| ReMaxBackbone=pythia-2.8B, Evaluator=Deberta-v3-large-v22025.12 | 62.7 | |
| GRPOModel Family=Pythia, Size=1B2026.02 | 62.2 | |
| GRPOModel Family=Qwen 2.5, Size=0.5B2026.02 | 54.1 | |
| GRPO + R2M w/o TrainBase Model=Pythia-2.8B, Variant=Without training2026.01 | 51.1 | |
| RLOO + R2M w/o TrainBase Model=Pythia-2.8B, Variant=Without training2026.01 | 50.6 | |
| TR-DPOModel Family=Qwen 2.5, Size=0.5B2026.02 | 49.7 | |
| DPOModel Family=Qwen 2.5, Size=0.5B2026.02 | 48.5 | |
| DPOModel Family=Pythia, Size=1B2026.02 | 45.9 | |
| TR-DPOModel Family=Pythia, Size=1B2026.02 | 45.5 | |
| SFTBase Model=Pythia-2.8B2026.01 | 42.3 | |
| SFTModel Family=Qwen 2.5, Size=1.5B2026.02 | 28.1 | |
| SFTModel Family=Pythia, Size=2.8B2026.02 | 25.6 | |
| SFTModel Family=Pythia, Size=1B2026.02 | 17.8 | |
| SFTModel Family=Qwen 2.5, Size=0.5B2026.02 | 16.4 |