Mathematics Question Answering on GSM8K (test, zero-shot)
80.67Flexible EMOriginal
Evaluation results
| Method | Links | ||
|---|---|---|---|
| OriginalModel=LLaMA3-70b2026.02 | 80.67 | 80.52 | |
| CIS (ours)Model=LLaMA3-70b, ρ̂=0.183, Theoretical Complexity (Comp*)=0.0117T2026.02 | 76.13 | 75.92 | |
| CIS* (ours)Model=LLaMA3-70b, ρ̂=0.179, Theoretical Complexity (Comp*)=0.0114T2026.02 | 75.84 | 75.56 | |
| CIS (ours)Model=LLaMA3-70b, ρ̂=0.082, Theoretical Complexity (Comp*)=0.0052T2026.02 | 75.53 | 75.26 | |
| CIS* (ours)Model=LLaMA3-70b, ρ̂=0.079, Theoretical Complexity (Comp*)=0.0050T2026.02 | 75.42 | 75.27 | |
| CIS* (ours)Model=LLaMA3-70b, ρ̂=0.061, Theoretical Complexity (Comp*)=0.0038T2026.02 | 75.36 | 75.04 | |
| CIS (ours)Model=LLaMA3-70b, ρ̂=0.064, Theoretical Complexity (Comp*)=0.0041T2026.02 | 75.28 | 75.11 | |
| Hshare-1Model=LLaMA3-70b, ρ̂=0.281, Theoretical Complexity (Comp*)=0.0180T2026.02 | 75.12 | 74.37 | |
| DSModel=LLaMA3-70b, Theoretical Complexity (Comp*)=0.0625T2026.02 | 74.15 | 73.92 | |
| Hshare-2Model=LLaMA3-70b, ρ̂=0.125, Theoretical Complexity (Comp*)=0.0080T2026.02 | 74.15 | 73.46 | |
| OriginalModel=Mistral-7b2026.02 | 38.21 | 38.13 | |
| CIS (ours)Model=Mistral-7b, ρ̂=0.194, Theoretical Complexity (Comp*)=0.0124T2026.02 | 33.25 | 33.02 | |
| CIS* (ours)Model=Mistral-7b, ρ̂=0.189, Theoretical Complexity (Comp*)=0.0121T2026.02 | 32.92 | 32.57 | |
| CIS (ours)Model=Mistral-7b, ρ̂=0.088, Theoretical Complexity (Comp*)=0.0056T2026.02 | 32.81 | 32.75 | |
| DSModel=Mistral-7b, Theoretical Complexity (Comp*)=0.0625T2026.02 | 32.62 | 32.27 | |
| CIS (ours)Model=Mistral-7b, ρ̂=0.065, Theoretical Complexity (Comp*)=0.0041T2026.02 | 32.33 | 32.01 | |
| CIS* (ours)Model=Mistral-7b, ρ̂=0.091, Theoretical Complexity (Comp*)=0.0058T2026.02 | 32.16 | 32.04 | |
| CIS* (ours)Model=Mistral-7b, ρ̂=0.068, Theoretical Complexity (Comp*)=0.0043T2026.02 | 32.04 | 31.75 | |
| Hshare-1Model=Mistral-7b, ρ̂=0.281, Theoretical Complexity (Comp*)=0.0180T2026.02 | 31.99 | 31.54 | |
| Hshare-2Model=Mistral-7b, ρ̂=0.125, Theoretical Complexity (Comp*)=0.0080T2026.02 | 28.18 | 26.84 | |
| OriginalModel=LLaMA2-7b-chat2026.02 | 22.97 | 22.97 | |
| CIS (ours)Model=LLaMA2-7b-chat, ρ̂=0.164, Theoretical Complexity (Comp*)=0.0104T2026.02 | 18.42 | 18.27 | |
| CIS* (ours)Model=LLaMA2-7b-chat, ρ̂=0.161, Theoretical Complexity (Comp*)=0.0103T2026.02 | 18.12 | 18.04 | |
| CIS (ours)Model=LLaMA2-7b-chat, ρ̂=0.087, Theoretical Complexity (Comp*)=0.0055T2026.02 | 18.04 | 17.82 | |
| Hshare-1Model=LLaMA2-7b-chat, ρ̂=0.281, Theoretical Complexity (Comp*)=0.0180T2026.02 | 18.03 | 17.03 | |
| CIS* (ours)Model=LLaMA2-7b-chat, ρ̂=0.084, Theoretical Complexity (Comp*)=0.0047T2026.02 | 17.89 | 17.82 | |
| CIS (ours)Model=LLaMA2-7b-chat, ρ̂=0.062, Theoretical Complexity (Comp*)=0.0039T2026.02 | 17.59 | 17.51 | |
| CIS* (ours)Model=LLaMA2-7b-chat, ρ̂=0.063, Theoretical Complexity (Comp*)=0.0040T2026.02 | 17.36 | 17.29 | |
| DSModel=LLaMA2-7b-chat, Theoretical Complexity (Comp*)=0.0625T2026.02 | 17.13 | 17.06 | |
| Hshare-2Model=LLaMA2-7b-chat, ρ̂=0.125, Theoretical Complexity (Comp*)=0.0080T2026.02 | 15.24 | 11.45 | |
| H2OModel=LLaMA2-7b-chat, Theoretical Complexity (Comp*)=O(1)2026.02 | 9.86 | 1.44 | |
| QuestModel=LLaMA2-7b-chat, Theoretical Complexity (Comp*)=0.1250T2026.02 | 4.78 | 4.62 |