Multi-task numerical and arithmetic reasoning on BitTokens benchmark
99.9Exact Match Acc (Min/Max)BitToken
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| BitTokenNumber encoding strategy=BitToken2025.10 | 99.9 | 100 | 99.8 | 0.989 | 0.991 | 0.989 | |
| SubwordNumber encoding strategy=Subword2025.10 | 99.1 | 99.3 | 92.7 | 0.954 | 0.45 | 0.522 | |
| Single DigitNumber encoding strategy=Single Digit2025.10 | 99.1 | 99.2 | 95.5 | 0.974 | 0.893 | 0.681 | |
| FoNENumber encoding strategy=FoNE2025.10 | 85.9 | 99.7 | 57.4 | 0.809 | 0.3 | 0.275 | |
| xValNumber encoding strategy=xVal2025.10 | 0.1 | 32.4 | 0 | 0.092 | 0.094 | 0.093 |