Mathematical Reasoning on GSM-Symbolic
82.1GSM-Sym AccuracyLatent Thinking Optimization
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Latent Thinking OptimizationModel=Llama-3-8B2025.09 | 82.1 | — | — | |
| Majority VotingModel=Llama-3-8B2025.09 | 79.6 | — | — | |
| Opt-Sym (+MATH)RL Method=GRPO, Data type=Opt-Sym (+MATH)2026.02 | 78.22 | 65.78 | 38 | |
| Opt-SymRL Method=GRPO, Data type=Opt-Sym2026.02 | 77.52 | 65.58 | 35.4 | |
| Base-Sym (+MATH)RL Method=GRPO, Data type=Base-Sym (+MATH)2026.02 | 77.36 | 63.92 | 33.96 | |
| Opt-SymRL Method=PPO, Data type=Opt-Sym2026.02 | 76.7 | 64.94 | 35.36 | |
| Base-NLRL Method=PPO, Data type=Base-NL2026.02 | 76.22 | 64.1 | 35.68 | |
| Opt-NLRL Method=PPO, Data type=Opt-NL2026.02 | 76.04 | 62.64 | 34.88 | |
| Base-SymRL Method=GRPO, Data type=Base-Sym2026.02 | 75.56 | 63.6 | 33.96 | |
| Base-NLRL Method=GRPO, Data type=Base-NL2026.02 | 75.08 | 63.7 | 32.48 | |
| Base-SymRL Method=PPO, Data type=Base-Sym2026.02 | 74.74 | 64.38 | 36.64 | |
| Opt-NLRL Method=GRPO, Data type=Opt-NL2026.02 | 74.22 | 63.88 | 34.48 | |
| Base ModelModel=Llama-3-8B2025.09 | 73.6 | — | — | |
| Seed-dataRL Method=GRPO, Data type=Seed-data2026.02 | 73.04 | 61.54 | 32.64 | |
| Seed-dataRL Method=PPO, Data type=Seed-data2026.02 | 72.96 | 61.12 | 32.16 | |
| Seed-data (+MATH)RL Method=GRPO, Data type=Seed-data (+MATH)2026.02 | 72.54 | 60.4 | 30.24 | |
| Galactica-6.7B (PPO)lambda=0.5, mu=0.5, rho=0.0, voting=true2026.04 | 68.12 | 57.28 | 28.36 | |
| Galactica-6.7B (PPO)lambda=1.0, mu=0.0, rho=0.0, voting=true2026.04 | 67.42 | 57.94 | 27 | |
| Galactica-6.7B (PPO)lambda=0.5, mu=0.5, rho=0.0, voting=false2026.04 | 66.32 | 53.32 | 25.72 | |
| Galactica-6.7B (PPO)lambda=1.0, mu=0.0, rho=0.0, voting=false2026.04 | 64.24 | 54.82 | 23.36 | |
| Qwen2.5-1.5B-InstructRL Method=None, Data type=None2026.02 | 64.16 | 49.98 | 23.48 | |
| DCCDModel Size=14B, Backbone=Qwen 2.5 14B2026.02 | 53 | — | — | |
| CFModel Size=14B, Backbone=Qwen 2.5 14B2026.02 | 49 | — | — | |
| Latent Thinking OptimizationModel=Mistral-7B2025.09 | 46.2 | — | — | |
| Latent Thinking OptimizationModel=Llama-2-13B2025.09 | 44.2 | — | — | |
| Latent Thinking OptimizationModel=Llama-2-13B2025.09 | 44.2 | — | — | |
| CPModel Size=14B, Backbone=Qwen 2.5 14B2026.02 | 44 | — | — | |
| Majority VotingModel=Mistral-7B2025.09 | 41.3 | — | — | |
| DCCDModel Size=7B, Backbone=Qwen 2.5 7B2026.02 | 41 | — | — | |
| Majority VotingModel=Llama-2-13B2025.09 | 37.9 | — | — | |
| Majority VotingModel=Llama-2-13B2025.09 | 37.9 | — | — | |
| CDModel Size=14B, Backbone=Qwen 2.5 14B2026.02 | 37 | — | — | |
| DCCDModel Size=3B, Backbone=Qwen 2.5 3B2026.02 | 36 | — | — | |
| Galactica-125M (GRPO)lambda=0.5, mu=0.5, rho=0.0, voting=false2026.04 | 35.36 | 11.18 | 2.2 | |
| Latent Thinking OptimizationModel=Llama-2-7B2025.09 | 31.6 | — | — | |
| Latent Thinking OptimizationModel=Llama-2-7B2025.09 | 31.6 | — | — | |
| CPModel Size=7B, Backbone=Qwen 2.5 7B2026.02 | 31 | — | — | |
| Latent Thinking Optimization w. LRMModel=Huginn-3.5B2025.09 | 30.5 | — | — | |
| Majority VotingModel=Llama-2-7B2025.09 | 30.2 | — | — | |
| Majority VotingModel=Llama-2-7B2025.09 | 30.2 | — | — | |
| Weighted Majority Voting w. LRMModel=Huginn-3.5B2025.09 | 30.1 | — | — | |
| DCCDModel Size=8B, Backbone=Llama 3.1 8B2026.02 | 30 | — | — | |
| CFModel Size=7B, Backbone=Qwen 2.5 7B2026.02 | 29 | — | — | |
| Self-Correction w. Confidence ScoreModel=Huginn-3.5B2025.09 | 28.1 | — | — | |
| Base ModelModel=Mistral-7B2025.09 | 27.8 | — | — | |
| Base ModelModel=Llama-2-13B2025.09 | 27.3 | — | — | |
| Base ModelModel=Llama-2-13B2025.09 | 27.3 | — | — | |
| Majority VotingModel=Huginn-3.5B2025.09 | 26.9 | — | — | |
| Base ModelModel=Huginn-3.5B2025.09 | 26.5 | — | — | |
| Galactica-125M (GRPO)lambda=1.0, mu=0.0, rho=0.0, voting=false2026.04 | 26.3 | 7.98 | 1.36 | |
| CDModel Size=7B, Backbone=Qwen 2.5 7B2026.02 | 26 | — | — | |
| Latent Thinking Correction w. CoE-RModel=Huginn-3.5B2025.09 | 25.9 | — | — | |
| Latent Thinking Correction w. CoE-CModel=Huginn-3.5B2025.09 | 25.6 | — | — | |
| CFModel Size=3B, Backbone=Qwen 2.5 3B2026.02 | 25 | — | — | |
| CFModel Size=1.5B, Backbone=Qwen 2.5 1.5B2026.02 | 23 | — | — | |
| DCCDModel Size=1.5B, Backbone=Qwen 2.5 1.5B2026.02 | 23 | — | — | |
| Base ModelModel=Llama-2-7B2025.09 | 20.4 | — | — | |
| Base ModelModel=Llama-2-7B2025.09 | 20.4 | — | — | |
| Self-Correction w. Verbal EvaluationModel=Huginn-3.5B2025.09 | 19.3 | — | — | |
| CPModel Size=3B, Backbone=Qwen 2.5 3B2026.02 | 19 | — | — | |
| CDModel Size=8B, Backbone=Llama 3.1 8B2026.02 | 19 | — | — | |
| CDModel Size=3B, Backbone=Qwen 2.5 3B2026.02 | 17 | — | — | |
| CPModel Size=8B, Backbone=Llama 3.1 8B2026.02 | 17 | — | — | |
| Latent Thinking OptimizationModel=OLMo-7B2025.09 | 15.4 | — | — | |
| Majority VotingModel=OLMo-7B2025.09 | 14.9 | — | — | |
| CFModel Size=8B, Backbone=Llama 3.1 8B2026.02 | 14 | — | — | |
| CDModel Size=1.5B, Backbone=Qwen 2.5 1.5B2026.02 | 12 | — | — | |
| CPModel Size=1.5B, Backbone=Qwen 2.5 1.5B2026.02 | 11 | — | — | |
| CFModel Size=1B, Backbone=Llama 3.2 1B2026.02 | 9 | — | — | |
| DCCDModel Size=1B, Backbone=Llama 3.2 1B2026.02 | 9 | — | — | |
| Base ModelModel=OLMo-7B2025.09 | 7.8 | — | — | |
| CPModel Size=1B, Backbone=Llama 3.2 1B2026.02 | 6 | — | — | |
| CDModel Size=1B, Backbone=Llama 3.2 1B2026.02 | 0 | — | — |