Mathematical Reasoning on MultiArith, GSM8K, AddSub, AQUA, SingleEq, SVAMP, and MAWPS (test)
99.7MultiArithS2FT
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| S2FTModel=LLAMA3-8B, #Param(%)=0.72024.12 | 99.7 | 65.8 | 93.7 | 31.5 | 97.8 | 76 | 92.4 | 79.6 | |
| LoRAModel=LLAMA3-8B, #Param(%)=0.72024.12 | 99.5 | 61.6 | 92.7 | 25.6 | 96.3 | 73.8 | 90.8 | 77.2 | |
| Full FTModel=LLaMA2-7B, #Param(%)=1002024.12 | 99.3 | 47.5 | 91.1 | 24.4 | 96.7 | 62.5 | 89.1 | 72.9 | |
| Full FTModel=LLAMA3-8B, #Param(%)=1002024.12 | 99.2 | 62 | 93.9 | 26.8 | 96.7 | 74 | 91.2 | 77.7 | |
| Full FTModel=LLaMA-7B, #Param(%)=1002024.12 | 98.8 | 43.1 | 91.1 | 20.9 | 94.3 | 60.6 | 88.2 | 71 | |
| S2FTModel=LLaMA-7B, #Param(%)=0.812024.12 | 98.8 | 41.3 | 91.4 | 21.3 | 93.5 | 58.4 | 86.1 | 70.1 | |
| DoRAModel=LLAMA3-8B, #Param(%)=0.712024.12 | 98.8 | 62.7 | 92.2 | 26.8 | 96.9 | 74 | 91.2 | 77.5 | |
| S2FTModel=LLaMA2-7B, #Param(%)=0.812024.12 | 98.5 | 44.3 | 91.1 | 25.2 | 94.7 | 61.8 | 88.2 | 72 | |
| Full FTModel=LLaMA-7B, Best Rank=–2025.06 | 98.5 | 42.68 | 91.9 | 22.83 | 95.08 | 59.5 | 89.5 | 71.43 | |
| S2FTModel=LLaMA-7B, Best Rank=1282025.06 | 98.5 | 40.49 | 92.15 | 24.8 | 95.28 | 57.2 | 89.5 | 71.13 | |
| Full FTModel=LLaMA-13B, #Param(%)=1002024.12 | 98.3 | 47.6 | 92.9 | 26 | 95.1 | 65.7 | 88.7 | 73.5 | |
| DoRAModel=LLaMA2-7B, #Param(%)=0.842024.12 | 98.2 | 43.8 | 90.1 | 24.4 | 94.5 | 59.1 | 89.1 | 71.3 | |
| PiSSAModel=LLaMA-7B, Best Rank=1282025.06 | 98.17 | 41.77 | 92.66 | 25.98 | 96.26 | 59.5 | 87.81 | 71.74 | |
| DoRAModel=LLaMA-7B, Best Rank=642025.06 | 98.17 | 43.59 | 91.9 | 23.23 | 95.08 | 60.1 | 86.13 | 71.17 | |
| LIFTModel=LLaMA-7B, Best Rank=2562025.06 | 98.17 | 42.15 | 92.41 | 23.62 | 95.47 | 62.7 | 87.82 | 71.76 | |
| LoRAModel=LLaMA-7B, #Param(%)=0.832024.12 | 98 | 40 | 91.2 | 21.7 | 93.1 | 56.7 | 85.3 | 69.7 | |
| S2FTModel=LLaMA-13B, #Param(%)=0.652024.12 | 97.7 | 48.4 | 90.4 | 22.8 | 95.5 | 63.9 | 87.8 | 72.4 | |
| LoRAModel=LLaMA-7B, Best Rank=642025.06 | 97.67 | 43.29 | 90.38 | 22.83 | 95.08 | 60.8 | 84.87 | 70.7 | |
| LoRAModel=LLaMA-13B, #Param(%)=0.672024.12 | 97.5 | 47.8 | 89.9 | 20.5 | 94.3 | 61.2 | 87.4 | 71.2 | |
| LoRAModel=LLaMA2-7B, #Param(%)=0.832024.12 | 97.5 | 44 | 91.2 | 20.9 | 94.1 | 59.2 | 85.7 | 70.4 | |
| DoRAModel=LLaMA-7B, #Param(%)=0.842024.12 | 97.3 | 38.9 | 89.6 | 22.4 | 93.9 | 58.4 | 85.3 | 69.4 | |
| DoRAModel=LLaMA-13B, #Param(%)=0.682024.12 | 97.2 | 48.1 | 90.6 | 20.9 | 93.9 | 63.8 | 88.2 | 71.8 | |
| GPT-3.5Model=GPT-3.52024.12 | 83.8 | 56.4 | 85.3 | 38.9 | 88.1 | 69.9 | 87.4 | 72.8 |