Mathematical & Scientific Reasoning on GPQA
68.54AccuracyGRPO
Evaluation results
| Method | Links | |
|---|---|---|
| GRPOSampling protocol=@162026.05 | 68.54 | |
| RDPOSampling protocol=@162026.05 | 67.79 | |
| RL initialization modelSampling protocol=@162026.05 | 66.36 | |
| Qwen3-8BEvaluation Protocol=Zero-shot, TPS=30.432025.12 | 39.06 | |
| REFUSIONEvaluation Protocol=Zero-shot, TPS=64.112025.12 | 35.49 | |
| LLaDA-8B-InstructEvaluation Protocol=Zero-shot, TPS=1.992025.12 | 32.37 | |
| LLaDA w/ D2FEvaluation Protocol=Zero-shot, TPS=23.842025.12 | 31.25 | |
| Dream w/ Fast-dLLMEvaluation Protocol=Zero-shot, TPS=22.962025.12 | 31.25 | |
| Dream-7B-InstructEvaluation Protocol=Zero-shot, TPS=1.812025.12 | 30.36 | |
| Dream w/ D2FEvaluation Protocol=Zero-shot, TPS=49.962025.12 | 30.13 | |
| Llama-3-8B-InstructEvaluation Protocol=Zero-shot, TPS=422025.12 | 29.46 | |
| LLaDA w/ Fast-dLLMEvaluation Protocol=Zero-shot, TPS=17.542025.12 | 28.35 | |
| MemGenBackbone=Llama 3.1 8B Instruct2026.01 | 18.43 | |
| FlashMemBackbone=Llama 3.1 8B Instruct2026.01 | 17.86 | |
| CoT-SCBackbone=Llama 3.1 8B Instruct2026.01 | 17.8 | |
| SnapKVBackbone=Llama 3.1 8B Instruct2026.01 | 15.66 | |
| VanillaBackbone=Llama 3.1 8B Instruct2026.01 | 15.48 | |
| MemGenBackbone=Llama 3.2 3B Instruct2026.01 | 7.07 | |
| FlashMemBackbone=Llama 3.2 3B Instruct2026.01 | 6.7 | |
| SnapKVBackbone=Llama 3.2 3B Instruct2026.01 | 5.27 | |
| CoT-SCBackbone=Llama 3.2 3B Instruct2026.01 | 5.18 | |
| VanillaBackbone=Llama 3.2 3B Instruct2026.01 | 4.79 |