Mathematical Reasoning on MATH500 (Pass@3)
76Pass@3 AccuracyT*
Evaluation Results
| Method | Links | |
|---|---|---|
| T*Base model=SDAR-4B-Chat-b8, Block size=b82026.01 | 76 | |
| TRACERLBase model=SDAR-4B-Chat-b4, Block size=b42026.01 | 75.33 | |
| SDAR-4B-Chat-b4Method=Original Checkpoint, Block size=b42026.01 | 68.67 | |
| T*Base model=SDAR-4B-Chat-b16, Block size=b162026.01 | 64.53 | |
| T*Base model=SDAR-1.7B-Chat-b8, Block size=b82026.01 | 63.4 | |
| TRACERLBase model=SDAR-1.7B-Chat-b4, Block size=b42026.01 | 62.53 | |
| TRACERLBase model=SDAR-4B-Chat-b8, Block size=b82026.01 | 62.1 | |
| SDAR-1.7B-Chat-b4Method=Original Checkpoint, Block size=b42026.01 | 61.4 | |
| SDAR-4B-Chat-b8Method=Original Checkpoint, Block size=b82026.01 | 60.73 | |
| TRACERLBase model=SDAR-4B-Chat-b16, Block size=b162026.01 | 60.5 | |
| TRACERLBase model=SDAR-1.7B-Chat-b8, Block size=b82026.01 | 60.2 | |
| T*Base model=SDAR-1.7B-Chat-b16, Block size=b162026.01 | 59.8 | |
| T*Base model=SDAR-1.7B-Chat-b32, Block size=b322026.01 | 59 | |
| SDAR-4B-Chat-b16Method=Original Checkpoint, Block size=b162026.01 | 58.26 | |
| SDAR-1.7B-Chat-b8Method=Original Checkpoint, Block size=b82026.01 | 55.9 | |
| TRACERLBase model=SDAR-1.7B-Chat-b16, Block size=b162026.01 | 54.4 | |
| SDAR-1.7B-Chat-b32Method=Original Checkpoint, Block size=b322026.01 | 54.2 | |
| TRACERLBase model=SDAR-1.7B-Chat-b32, Block size=b322026.01 | 54.1 | |
| SDAR-1.7B-Chat-b16Method=Original Checkpoint, Block size=b162026.01 | 52.4 |