Mathematical Reasoning on MATHINSTRUCT Evaluation Suite (In-domain and Out-of-domain)
54.89GSM8K AccuracyUniPROT
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| UniPROTBackbone=ZEPHYR-3B, Selection Configuration=batch-wise, Batch Size=128, Total Budget=322026.04 | 54.89 | 19.6 | 40.4 | 38.5 | 54.1 | 15.5 | 22.95 | 31.34 | 34.92 | |
| UniPROTBackbone=ZEPHYR-3B, Selection Configuration=source-wise, Batch Size=128, Total Budget=322026.04 | 54.4 | 20.4 | 41.3 | 38.5 | 54.3 | 16.5 | 24.7 | 31.34 | 34.92 | |
| GREATSBackbone=ZEPHYR-3B, Selection Configuration=source-wise, Batch Size=128, Total Budget=322026.04 | 52.8 | 19.01 | 40.8 | 36.32 | 54.5 | 17.1 | 21.98 | 30.64 | 33.48 | |
| GradNormBackbone=ZEPHYR-3B, Selection Configuration=source-wise, Batch Size=128, Total Budget=322026.04 | 52.3 | 16.5 | 40 | 35.7 | 53.5 | 15.5 | 22.5 | 30.45 | 33.07 | |
| FTBackbone=ZEPHYR-3B, Batch Size=128, Total Budget=322026.04 | 52.08 | 16.14 | 40.1 | 36.11 | 54.7 | 15.6 | 22.2 | 30.83 | 33.47 | |
| SBERTBackbone=ZEPHYR-3B, Selection Configuration=source-wise, Batch Size=128, Total Budget=322026.04 | 51.3 | 14.6 | 41.6 | 28.44 | 56.3 | 16.2 | 21.98 | 26.06 | 27.25 | |
| COLMBackbone=ZEPHYR-3B, Selection Configuration=source-wise, Batch Size=128, Total Budget=322026.04 | 50.6 | 21.42 | 40.15 | 36.56 | 55.8 | 16.7 | 22.17 | 30.86 | 33.71 | |
| GREATSBackbone=ZEPHYR-3B, Selection Configuration=batch-wise, Batch Size=128, Total Budget=322026.04 | 50.6 | 15.9 | 38.8 | 36.32 | 54.6 | 13.9 | 21.78 | 30.64 | 33.48 | |
| COLMBackbone=ZEPHYR-3B, Selection Configuration=batch-wise, Batch Size=128, Total Budget=322026.04 | 49.88 | 17.4 | 39.9 | 36.56 | 54.1 | 15.4 | 21.01 | 30.86 | 33.71 | |
| GradNormBackbone=ZEPHYR-3B, Selection Configuration=batch-wise, Batch Size=128, Total Budget=322026.04 | 49.8 | 15.7 | 39.9 | 35.7 | 53.1 | 15.4 | 22.7 | 30.45 | 33.07 | |
| SBERTBackbone=ZEPHYR-3B, Selection Configuration=batch-wise, Batch Size=128, Total Budget=322026.04 | 21.7 | 14.01 | 27.44 | 28.44 | 34.5 | 13.6 | 13.8 | 26.06 | 27.25 |