Mathematical Reasoning on MathQA, GSM8K, AddSub, SingleEq, SVAMP
91.88MathQA Accuracy(IA)3
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| (IA)3LLM=LLAMA-2-13B, #TTPs=36.02M2024.10 | 91.88 | 79.24 | 88.16 | 87.08 | 65.63 | |
| RoCoFT-ColumnLLM=LLAMA-2-13B, #TTPs=24.88M2024.10 | 91.46 | 79.63 | 88.11 | 87.58 | 66.63 | |
| LoRALLM=LLAMA-2-13B, #TTPs=44.94M2024.10 | 91.27 | 78.9 | 86.89 | 86.07 | 65.85 | |
| RoCoFT-RowLLM=LLAMA-2-13B, #TTPs=24.88M2024.10 | 90.69 | 79.7 | 88.24 | 87.28 | 66.6 | |
| RoCoFT-RowLLM=LLAMA-2-7B, #TTPs=13.47M2024.10 | 90.55 | 77.37 | 86.12 | 82.66 | 60.75 | |
| AdaLoRALLM=LLAMA-2-13B, #TTPs=45.04M2024.10 | 90.55 | 80.19 | 87 | 87.1 | 66.03 | |
| RoCoFT-ColumnLLM=LLAMA-2-7B, #TTPs=13.47M2024.10 | 90.42 | 77.35 | 86.16 | 82.48 | 60.35 | |
| (IA)3LLM=LLAMA-2-7B, #TTPs=19.42M2024.10 | 90.2 | 76.13 | 86.55 | 83.7 | 59.16 | |
| LoRALLM=LLAMA-2-7B, #TTPs=24.30M2024.10 | 90.13 | 75.68 | 84.67 | 82.14 | 59.94 | |
| AdaLoRALLM=LLAMA-2-7B, #TTPs=24.90M2024.10 | 89.81 | 77.07 | 86.7 | 83.01 | 60.25 | |
| RoCoFT-ColumnLLM=GPT-J-6B, #TTPs=11.62M2024.10 | 89.74 | 72.24 | 80.23 | 82.61 | 56.7 | |
| (IA)3LLM=GPT-J-6B, #TTPs=16.61M2024.10 | 89.51 | 72.04 | 80.5 | 81.5 | 55.43 | |
| RoCoFT-RowLLM=GPT-J-6B, #TTPs=11.62M2024.10 | 89.45 | 72.8 | 80.45 | 82.12 | 56.79 | |
| AdaLoRALLM=GPT-J-6B, #TTPs=20.77M2024.10 | 88.51 | 72.45 | 80.21 | 82.03 | 56.14 | |
| LoRALLM=GPT-J-6B, #TTPs=20.02M2024.10 | 88.3 | 72.82 | 80.6 | 81.24 | 56.73 | |
| PrefixLLM=LLAMA-2-13B, #TTPs=61.97M2024.10 | 87.22 | 71.09 | 84.09 | 81.28 | 58.25 | |
| PrefixLLM=LLAMA-2-7B, #TTPs=33.53M2024.10 | 84.18 | 68.47 | 81.04 | 80 | 52.17 | |
| PrefixLLM=GPT-J-6B, #TTPs=27.83M2024.10 | 83.12 | 67.44 | 75.25 | 78.46 | 49.12 | |
| RoCoFT-RowLLM=BLOOMZ-7B, #TTPs=13.37M2024.10 | 79.76 | 70.94 | 70.95 | 70.9 | 54.42 | |
| RoCoFT-ColumnLLM=BLOOMZ-7B, #TTPs=13.37M2024.10 | 79.72 | 71.05 | 70.88 | 70.76 | 54.38 | |
| LoRALLM=BLOOMZ-7B, #TTPs=24.22M2024.10 | 79.5 | 70.93 | 70.9 | 70.59 | 53.85 | |
| AdaLoRALLM=BLOOMZ-7B, #TTPs=24.88M2024.10 | 79.42 | 70.25 | 68.93 | 70.93 | 53.89 | |
| (IA)3LLM=BLOOMZ-7B, #TTPs=19.34M2024.10 | 78.9 | 71.17 | 70.33 | 70.84 | 53.95 | |
| PrefixLLM=BLOOMZ-7B, #TTPs=33.37M2024.10 | 78.41 | 66.45 | 67.52 | 66.94 | 49.1 |