Multi-Task Evaluation on Countdown, GSM8K, OlympiadBench, ROCStories, and MBPP (test)
56.3Overall ScoreRandOpt K=50
Evaluation results
| Method | Links | |
|---|---|---|
| RandOpt K=50Training cost (forward-pass-equivalent FLOPs)=1×10⁶, Inference cost (forward passes per test example)=502026.05 | 56.3 | |
| CoRPTraining cost (forward-pass-equivalent FLOPs)=1×10⁵, Inference cost (forward passes per test example)=12026.05 | 50 | |
| GRPOTraining cost (forward-pass-equivalent FLOPs)=1×10⁶, Inference cost (forward passes per test example)=12026.05 | 48.8 | |
| PPOTraining cost (forward-pass-equivalent FLOPs)=1×10⁶, Inference cost (forward passes per test example)=12026.05 | 47.7 | |
| RandOpt K=1Training cost (forward-pass-equivalent FLOPs)=1×10⁶, Inference cost (forward passes per test example)=12026.05 | 43.5 | |
| BaseTraining cost (forward-pass-equivalent FLOPs)=–, Inference cost (forward passes per test example)=12026.05 | 41.9 |