Mathematical Reasoning on Countdown (Context Lengths 128, 256, 512)
54.3Accuracy (L=128)AIS
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| AISBackbone=LLaDA-8B, Evaluation Protocol=Full-model fine-tuning2026.05 | 54.3 | 57.03 | 54.69 | — | |
| BF16Backbone=LLaDA-8B, Evaluation Protocol=Full-model fine-tuning2026.05 | 44.8 | 52 | 52.2 | — | |
| EvoToken (10k FT)NFE / Gen Len=12026.01 | 39.84 | 35.55 | 42.97 | 39.45 | |
| FlashRL (TIS)Backbone=LLaDA-8B, Evaluation Protocol=Full-model fine-tuning2026.05 | 39.06 | 42.97 | 48.05 | — | |
| FP8 RolloutBackbone=LLaDA-8B, Evaluation Protocol=Full-model fine-tuning2026.05 | 37.73 | 34.38 | 33.2 | — | |
| EvoToken (10k FT)NFE / Gen Len=1/22026.01 | 34.77 | 30.08 | 30.08 | 31.64 | |
| FT-Base (10k FT)NFE / Gen Len=12026.01 | 33.2 | 21.48 | 19.53 | 24.74 | |
| FT-Base (10k FT)NFE / Gen Len=1/22026.01 | 28.12 | 16.8 | 16.41 | 20.44 | |
| BaselineNFE / Gen Len=1/22026.01 | 26.17 | 16.41 | 16.8 | 19.79 | |
| EvoToken (10k FT)NFE / Gen Len=1/42026.01 | 23.05 | 16.02 | 12.11 | 17.06 | |
| BaselineNFE / Gen Len=12026.01 | 21.48 | 23.83 | 20.7 | 22 | |
| BaselineNFE / Gen Len=1/42026.01 | 17.19 | 15.62 | 16.41 | 16.41 | |
| FT-Base (10k FT)NFE / Gen Len=1/42026.01 | 14.06 | 13.67 | 9.77 | 12.5 |