Reasoning on GSM8K, MATH500, HumanEval, and MBPP Suite
60.3Average AccuracyInfo-Gain
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Info-GainModel=SDAR-8B-Chat, K (Decoding Rate)=12026.02 | 60.3 | 41 | |
| Info-GainModel=TraDo-8B-Instruct, K (Decoding Rate)=12026.02 | 60.3 | 52.1 | |
| Info-GainModel=SDAR-8B-Chat, K (Decoding Rate)=22026.02 | 55.8 | 74.1 | |
| Info-GainModel=TraDo-8B-Instruct, K (Decoding Rate)=22026.02 | 55.3 | 98 | |
| KLASSModel=SDAR-8B-Chat, K (Decoding Rate)=12026.02 | 40 | 105.3 | |
| KLASSModel=TraDo-8B-Instruct, K (Decoding Rate)=12026.02 | 40 | 180.3 | |
| ConfidenceModel=SDAR-8B-Chat, K (Decoding Rate)=12026.02 | 36.9 | 117.4 | |
| KLASSModel=SDAR-8B-Chat, K (Decoding Rate)=22026.02 | 35 | 210.3 | |
| KLASSModel=TraDo-8B-Instruct, K (Decoding Rate)=22026.02 | 35 | 334.3 | |
| EntropyModel=SDAR-8B-Chat, K (Decoding Rate)=12026.02 | 34.9 | 120.4 | |
| ConfidenceModel=TraDo-8B-Instruct, K (Decoding Rate)=12026.02 | 34.3 | 163.5 | |
| MarginModel=SDAR-8B-Chat, K (Decoding Rate)=12026.02 | 33 | 138.2 | |
| ConfidenceModel=SDAR-8B-Chat, K (Decoding Rate)=22026.02 | 32.1 | 204.1 | |
| ConfidenceModel=TraDo-8B-Instruct, K (Decoding Rate)=22026.02 | 29.3 | 334.1 | |
| EntropyModel=SDAR-8B-Chat, K (Decoding Rate)=22026.02 | 28.4 | 238.6 | |
| EntropyModel=TraDo-8B-Instruct, K (Decoding Rate)=12026.02 | 27.8 | 171.4 | |
| MarginModel=TraDo-8B-Instruct, K (Decoding Rate)=12026.02 | 27.7 | 208 | |
| MarginModel=SDAR-8B-Chat, K (Decoding Rate)=22026.02 | 26.7 | 230.9 | |
| EntropyModel=TraDo-8B-Instruct, K (Decoding Rate)=22026.02 | 22.8 | 419.5 | |
| MarginModel=TraDo-8B-Instruct, K (Decoding Rate)=22026.02 | 22.7 | 398 |