Mathematical Reasoning on GSM8K (test) (EM Accuracy, Token Counts, Pruning Inference)
79.08EM AccuracyBase
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| BaseBackbone=Qwen2.5-7B2026.05 | 79.08 | — | — | |
| TMAPBackbone=Qwen2.5-7B2026.05 | 79 | — | — | |
| Data-WhispererBackbone=Qwen2.5-7B2026.05 | 78.77 | — | — | |
| LongestBackbone=Qwen2.5-7B2026.05 | 78.7 | — | — | |
| IFDBackbone=Qwen2.5-7B2026.05 | 77.33 | — | — | |
| DiffuMaskshot_setting=32-shot, evaluation_type=In-domain2026.04 | 77.07 | 1,113.8 | 0.75 | |
| COT-INFLUX (CI)shot_setting=32-shot, evaluation_type=In-domain2026.04 | 76.9 | 1,644.4 | 0.5 | |
| AlpagasusBackbone=Qwen2.5-7B2026.05 | 76.57 | — | — | |
| Full (Uncompressed)shot_setting=32-shot, evaluation_type=In-domain2026.04 | 76.49 | 5,399.1 | — | |
| DEITABackbone=Qwen2.5-7B2026.05 | 76.27 | — | — | |
| Hierarchical Pruning (HP)shot_setting=32-shot, evaluation_type=In-domain2026.04 | 75.83 | 660.4 | 1,440 | |
| Full (100%)Backbone=Qwen2.5-7B2026.05 | 75.36 | — | — | |
| RandomBackbone=Qwen2.5-7B2026.05 | 75.28 | — | — | |
| BaseBackbone=Llama3-8B2026.05 | 72.1 | — | — | |
| LLMLingua2shot_setting=32-shot, evaluation_type=In-domain2026.04 | 70.89 | 1,155.2 | 0.1 | |
| LLMLinguashot_setting=32-shot, evaluation_type=In-domain2026.04 | 67.32 | 1,153.2 | 0.1 | |
| TMAPBackbone=Llama3-8B2026.05 | 65.96 | — | — | |
| Full (100%)Backbone=Llama3-8B2026.05 | 64.67 | — | — | |
| DEITABackbone=Llama3-8B2026.05 | 62.7 | — | — | |
| RandomBackbone=Llama3-8B2026.05 | 61.49 | — | — | |
| Randomshot_setting=32-shot, evaluation_type=In-domain2026.04 | 60.89 | 1,061.3 | 0 | |
| AlpagasusBackbone=Llama3-8B2026.05 | 60.88 | — | — | |
| Data-WhispererBackbone=Llama3-8B2026.05 | 60.73 | — | — | |
| LongestBackbone=Llama3-8B2026.05 | 60.65 | — | — | |
| IFDBackbone=Llama3-8B2026.05 | 60.02 | — | — | |
| Full (100%)Backbone=Mistral-v0.2-7B2026.05 | 58.61 | — | — | |
| TMAPBackbone=Mistral-v0.2-7B2026.05 | 55.42 | — | — | |
| AlpagasusBackbone=Mistral-v0.2-7B2026.05 | 52.77 | — | — | |
| S2Lshot=0-shot, fine-tuning=50% coreset from MATHINSTRUCT, backbone=LLAMA-2-7B2025.10 | 52.45 | — | — | |
| TRIMshot=0-shot, fine-tuning=50% coreset from MATHINSTRUCT, backbone=LLAMA-2-7B2025.10 | 52.23 | — | — | |
| LESSshot=0-shot, fine-tuning=50% coreset from MATHINSTRUCT, backbone=LLAMA-2-7B2025.10 | 52.1 | — | — | |
| IFDBackbone=Mistral-v0.2-7B2026.05 | 52.08 | — | — | |
| DEITABackbone=Mistral-v0.2-7B2026.05 | 52.01 | — | — | |
| LongestBackbone=Mistral-v0.2-7B2026.05 | 51.25 | — | — | |
| Full-data Fine-tuningshot=0-shot, fine-tuning=full-data, backbone=LLAMA-2-7B2025.10 | 51.15 | — | — | |
| RandomBackbone=Mistral-v0.2-7B2026.05 | 50.8 | — | — | |
| TAGCOSshot=0-shot, fine-tuning=50% coreset from MATHINSTRUCT, backbone=LLAMA-2-7B2025.10 | 50.2 | — | — | |
| Data-WhispererBackbone=Mistral-v0.2-7B2026.05 | 49.58 | — | — | |
| Randomshot=0-shot, fine-tuning=50% coreset from MATHINSTRUCT, backbone=LLAMA-2-7B2025.10 | 48.16 | — | — | |
| BaseBackbone=Mistral-v0.2-7B2026.05 | 38.67 | — | — | |
| Pretrained (no Fine-tuning)shot=0-shot, fine-tuning=none, backbone=LLAMA-2-7B2025.10 | 3.1 | — | — |