Long-form reasoning on NuminaProof (test)
-1.0172Avg LogProb (Per Answer)SFT (no CoT)
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| SFT (no CoT)Backbone=Qwen 3B, Reward algorithm=SFT (no CoT)2026.02 | -1.0172 | 2.77 | -1.0172 | 5 | |
| Log-probBackbone=Qwen 3B, Reward algorithm=Log-prob2026.02 | -1.0174 | 2.77 | -1.0174 | 5.3 | |
| Avg ProbabilityBackbone=Qwen 3B, Reward algorithm=Avg Probability2026.02 | -1.0217 | 2.78 | -1.0217 | 10 | |
| JEPOBackbone=Qwen 3B, Reward algorithm=JEPO2026.02 | -1.0218 | 2.78 | -1.0218 | 14.1 | |
| Avg LogprobBackbone=Qwen 3B, Reward algorithm=Avg Logprob2026.02 | -1.0235 | 2.78 | -1.0235 | 9 | |
| JEPOBackbone=Llama 3B, Reward algorithm=JEPO2026.02 | -1.1104 | 3.04 | -1.1102 | 34.1 | |
| Log-probBackbone=Llama 3B, Reward algorithm=Log-prob2026.02 | -1.1124 | 3.04 | -1.1124 | 9.1 | |
| SFT (no CoT)Backbone=Llama 3B, Reward algorithm=SFT (no CoT)2026.02 | -1.1127 | 3.04 | -1.1127 | 5 | |
| Avg ProbabilityBackbone=Llama 3B, Reward algorithm=Avg Probability2026.02 | -1.1157 | 3.05 | -1.1157 | 9.1 | |
| Avg LogprobBackbone=Llama 3B, Reward algorithm=Avg Logprob2026.02 | -1.1175 | 3.06 | -1.1175 | 9 | |
| Base modelBackbone=Qwen 3B, Reward algorithm=Base model2026.02 | -1.1838 | 3.27 | — | 225.6 | |
| ProbabilityBackbone=Qwen 3B, Reward algorithm=Probability2026.02 | -1.1862 | 3.27 | -1.1852 | 225.2 | |
| ProbabilityBackbone=Llama 3B, Reward algorithm=Probability2026.02 | -1.2859 | 3.62 | -1.285 | 469.6 | |
| Base modelBackbone=Llama 3B, Reward algorithm=Base model2026.02 | -1.2871 | 3.62 | — | 474 |