Science Reasoning on GPQA (Accuracy)
59.22Accuracy (GPQA)GSPO + NSR
Evaluation Results
| Method | Links | |
|---|---|---|
| GSPO + NSRModel=Qwen3-30B-A3B-Base, Zero-shot=true (k=32, T=1.0)2026.05 | 59.22 | |
| GSPOModel=Qwen3-30B-A3B-Base, Zero-shot=true (k=32, T=1.0)2026.05 | 58.96 | |
| In-place feedbackModel=Gemma-3-27B, Number of turns=42025.10 | 58.7 | |
| In-place feedbackModel=Llama-3.1-70B, Number of turns=42025.10 | 53.2 | |
| DAPO + NSRModel=Qwen3-8B-Base, Zero-shot=true (k=32, T=1.0)2026.05 | 53.16 | |
| DAPOModel=Qwen3-8B-Base, Zero-shot=true (k=32, T=1.0)2026.05 | 52.02 | |
| Multi-turnModel=Gemma-3-27B, Number of turns=42025.10 | 51.6 | |
| Multi-turnModel=Llama-3.1-70B, Number of turns=42025.10 | 50.8 | |
| MemoryModel=Llama-3.1-70B, Number of turns=42025.10 | 50.8 | |
| SoftThinkModel Size=4B*2025.11 | 50.3 | |
| OuroModel Size=4B*2025.11 | 50 | |
| In-place feedbackModel=Gemma-3-27B, Number of turns=32025.10 | 49.2 | |
| TaH+Model Size=4B*2025.11 | 49 | |
| TaHModel Size=4B*2025.11 | 48.5 | |
| Multi-turnModel=Gemma-3-27B, Number of turns=32025.10 | 48.4 | |
| MemoryModel=Gemma-3-27B, Number of turns=42025.10 | 47.6 | |
| In-place feedbackModel=Llama-3.1-70B, Number of turns=32025.10 | 46.8 | |
| IBTPOBackbone=Qwen3-14B-Base2026.05 | 46.6 | |
| MemoryModel=Llama-3.1-70B, Number of turns=32025.10 | 46.6 | |
| StandardModel Size=4B*2025.11 | 46.2 | |
| Multi-turnModel=Llama-3.1-70B, Number of turns=32025.10 | 46 | |
| IBROBackbone=Qwen3-14B-Base2026.05 | 44.7 | |
| MemoryModel=Gemma-3-27B, Number of turns=32025.10 | 44.4 | |
| GRPO w/ Entropy RegBackbone=Qwen3-14B-Base2026.05 | 44.2 | |
| TreeRLBackbone=Qwen3-14B-Base2026.05 | 44.2 | |
| Vanilla GRPOBackbone=Qwen3-14B-Base2026.05 | 43.7 | |
| In-place feedbackModel=Gemma-3-27B, Number of turns=22025.10 | 43.7 | |
| DAPO + NSRModel=Qwen2.5-Math-7B, Zero-shot=true (k=32, T=1.0)2026.05 | 42.99 | |
| In-place feedbackModel=Llama-3.1-70B, Number of turns=22025.10 | 42.9 | |
| Multi-turnModel=Gemma-3-27B, Number of turns=22025.10 | 42.9 | |
| Multi-turnModel=Llama-3.1-70B, Number of turns=22025.10 | 42.1 | |
| MemoryModel=Llama-3.1-70B, Number of turns=22025.10 | 40.5 | |
| MemoryModel=Gemma-3-27B, Number of turns=22025.10 | 39.7 | |
| TaH+Model Size=1.7B2025.11 | 39.4 | |
| DAPOModel=Qwen2.5-Math-7B, Zero-shot=true (k=32, T=1.0)2026.05 | 39.27 | |
| Initial ModelBackbone=Qwen3-14B-Base2026.05 | 38.6 | |
| OuroModel Size=0.6B2025.11 | 35.4 | |
| In-place feedbackModel=Gemma-3-27B, Number of turns=12025.10 | 34.9 | |
| Qwen3.5 2BN_act=1.9B, N_total=1.9B2026.05 | 34.3 | |
| Multi-turnModel=Gemma-3-27B, Number of turns=12025.10 | 34.1 | |
| MobileMoE-LN_act=922M, N_total=5.3B2026.05 | 33.8 | |
| SoftThinkModel Size=1.7B2025.11 | 33.3 | |
| TaHModel Size=1.7B2025.11 | 33.3 | |
| MemoryModel=Gemma-3-27B, Number of turns=12025.10 | 33.3 | |
| OuroModel Size=1.7B2025.11 | 32.8 | |
| In-place feedbackModel=Llama-3.1-70B, Number of turns=12025.10 | 31.8 | |
| TaH+Model Size=0.6B2025.11 | 31.3 | |
| StandardModel Size=0.6B2025.11 | 31.1 | |
| AlwaysThinkModel Size=0.6B2025.11 | 30.8 | |
| AlwaysThinkModel Size=1.7B2025.11 | 30.5 | |
| StandardModel Size=1.7B2025.11 | 30.3 | |
| MemoryModel=Llama-3.1-70B, Number of turns=12025.10 | 30.2 | |
| OLMo 2 1BN_act=1.5B, N_total=1.5B2026.05 | 29.8 | |
| SmolLM2 1.7BN_act=1.7B, N_total=1.7B2026.05 | 29.8 | |
| Multi-turnModel=Llama-3.1-70B, Number of turns=12025.10 | 29.4 | |
| TaHModel Size=0.6B2025.11 | 29 | |
| Llama 3.2 1BN_act=1.2B, N_total=1.2B2026.05 | 28.8 | |
| MobileMoE-SN_act=272M, N_total=1.3B2026.05 | 27.8 | |
| Qwen3.5 0.8BN_act=749M, N_total=749M2026.05 | 26.3 | |
| Gemma 3 270MN_act=270M, N_total=270M2026.05 | 25.8 | |
| SmolLM2 360MN_act=362M, N_total=362M2026.05 | 25.8 | |
| Gemma 3 1BN_act=1.0B, N_total=1.0B2026.05 | 25.3 | |
| MobileMoE-MN_act=528M, N_total=2.8B2026.05 | 24.8 | |
| SoftThinkModel Size=0.6B2025.11 | 24.7 | |
| OLMoE-1B-7BN_act=1.3B, N_total=6.9B2026.05 | 24.2 | |
| MobileLLM-ProN_act=1.1B, N_total=1.1B2026.05 | 23.2 | |
| Multi-turnModel=Llama-3.1-70B, Number of turns=02025.10 | 18.3 | |
| MemoryModel=Llama-3.1-70B, Number of turns=02025.10 | 18.3 | |
| In-place feedbackModel=Llama-3.1-70B, Number of turns=02025.10 | 18.3 | |
| Multi-turnModel=Gemma-3-27B, Number of turns=02025.10 | 17.5 | |
| MemoryModel=Gemma-3-27B, Number of turns=02025.10 | 17.5 | |
| In-place feedbackModel=Gemma-3-27B, Number of turns=02025.10 | 17.5 |