Mathematical Reasoning on GSM8K (Pass@3 accuracy)
91.2Pass@3 AccuracyTRACERL
Evaluation results
| Method | Links | |
|---|---|---|
| TRACERLBase model=SDAR-4B-Chat-b4, Block size=b42026.01 | 91.2 | |
| T*Base model=SDAR-4B-Chat-b8, Block size=b82026.01 | 91 | |
| SDAR-4B-Chat-b4Method=Original Checkpoint, Block size=b42026.01 | 90.5 | |
| T*Base model=SDAR-4B-Chat-b16, Block size=b162026.01 | 89.4 | |
| TRACERLBase model=SDAR-4B-Chat-b8, Block size=b82026.01 | 86.3 | |
| SDAR-4B-Chat-b8Method=Original Checkpoint, Block size=b82026.01 | 85.87 | |
| T*Base model=SDAR-1.7B-Chat-b8, Block size=b82026.01 | 82.4 | |
| TRACERLBase model=SDAR-1.7B-Chat-b4, Block size=b42026.01 | 82.23 | |
| T*Base model=SDAR-1.7B-Chat-b16, Block size=b162026.01 | 82.2 | |
| T*Base model=SDAR-1.7B-Chat-b32, Block size=b322026.01 | 82 | |
| TRACERLBase model=SDAR-1.7B-Chat-b8, Block size=b82026.01 | 81.5 | |
| SDAR-1.7B-Chat-b8Method=Original Checkpoint, Block size=b82026.01 | 81 | |
| SDAR-1.7B-Chat-b4Method=Original Checkpoint, Block size=b42026.01 | 80.54 | |
| TRACERLBase model=SDAR-1.7B-Chat-b16, Block size=b162026.01 | 80 | |
| TRACERLBase model=SDAR-1.7B-Chat-b32, Block size=b322026.01 | 79.8 | |
| TRACERLBase model=SDAR-4B-Chat-b16, Block size=b162026.01 | 79.6 | |
| SDAR-1.7B-Chat-b16Method=Original Checkpoint, Block size=b162026.01 | 79.4 | |
| SDAR-1.7B-Chat-b32Method=Original Checkpoint, Block size=b322026.01 | 78.31 | |
| SDAR-4B-Chat-b16Method=Original Checkpoint, Block size=b162026.01 | 78.24 |