Commonsense Reasoning on OBQA, ARC-C, Wino, PIQA, Social, ARC-E, BoolQ, Hella
94.8OBQAGenLoRA
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| GenLoRABackbone=Qwen2.5-14B, #Params=1.38M, Rank (r)=8, Group size (g)=82026.02 | 94.8 | 93.6 | 91.08 | 92.98 | 84.24 | 97.77 | 76.39 | 96.25 | 90.8 | |
| GenLoRABackbone=Qwen2.5-14B, #Params=2.26M, Rank (r)=32, Group size (g)=82026.02 | 94.8 | 94.2 | 90.77 | 93.31 | 84.44 | 98.11 | 76.79 | 96.36 | 91.1 | |
| HiRABackbone=Qwen2.5-14B, #Params=8.65M, Rank (r)=82026.02 | 94.2 | 93.26 | 89.98 | 93.09 | 83.88 | 97.98 | 75.87 | 96.5 | 90.6 | |
| DoRABackbone=Qwen2.5-14B, #Params=8.99M, Rank (r)=82026.02 | 93.2 | 92.75 | 84.85 | 92 | 81.73 | 97.35 | 74.71 | 95.37 | 88.99 | |
| LoRABackbone=Qwen2.5-14B, #Params=8.65M, Rank (r)=82026.02 | 93 | 92.66 | 84.85 | 91.89 | 81.47 | 97.35 | 74.71 | 95.33 | 88.91 | |
| AuroraBackbone=Qwen2.5-14B, #Params=8.65M, Rank (r)=82026.02 | 92.6 | 92.66 | 84.85 | 91.89 | 81.58 | 97.39 | 74.62 | 95.33 | 88.86 | |
| AuroraBackbone=Qwen2.5-14B, #Params=2.16M, Rank (r)=22026.02 | 90.6 | 91.21 | 78.22 | 90.86 | 80.19 | 97.05 | 73.49 | 93.81 | 86.93 | |
| MELoRABackbone=Qwen2.5-14B, #Params=8.65M, Rank (r)=82026.02 | 90.4 | 91.3 | 77.19 | 90.75 | 79.79 | 97.1 | 72.54 | 92.91 | 86.5 | |
| GenLoRABackbone=Gemma-7B, #Params=0.95M, Rank (r)=16, Group size (g)=82026.02 | 88 | 84.64 | 88.48 | 89.23 | 81.53 | 94.19 | 70.95 | 94.19 | 86.4 | |
| GenLoRABackbone=LLaMA-3-8B, #Params=2.16M, Rank (r)=32, Group size (g)=162026.02 | 86.8 | 80.12 | 85.71 | 88.74 | 80.91 | 91.37 | 72.63 | 92.87 | 84.89 | |
| HiRABackbone=Gemma-7B, #Params=4.82M, Rank (r)=82026.02 | 85.8 | 83.45 | 85 | 88.3 | 77.94 | 93.14 | 69.66 | 92.31 | 84.45 | |
| GenLoRABackbone=LLaMA-3-8B, #Params=0.98M, Rank (r)=8, Group size (g)=162026.02 | 85.6 | 79.01 | 84.85 | 88.57 | 79.38 | 91.75 | 71.83 | 91.45 | 84.05 | |
| GenLoRABackbone=Gemma-7B, #Params=0.77M, Rank (r)=8, Group size (g)=82026.02 | 84.8 | 80.8 | 86.9 | 87.54 | 77.43 | 92.26 | 71.25 | 92.07 | 84.13 | |
| HiRABackbone=LLaMA-3-8B, #Params=4.72M, Rank (r)=82026.02 | 84.4 | 80.29 | 85.4 | 89.01 | 79.43 | 91.96 | 65.44 | 92.17 | 83.51 | |
| DoRABackbone=Gemma-7B, #Params=5.16M, Rank (r)=82026.02 | 82.6 | 80.89 | 84.06 | 85.47 | 75.95 | 91.16 | 69.2 | 89.42 | 82.34 | |
| AuroraBackbone=Gemma-7B, #Params=4.82M, Rank (r)=82026.02 | 82.4 | 82.76 | 83.5 | 85.96 | 76.82 | 91.75 | 70.28 | 90.08 | 82.94 | |
| LoRABackbone=Gemma-7B, #Params=4.82M, Rank (r)=82026.02 | 81.8 | 82.76 | 86.19 | 88.3 | 77.99 | 92.55 | 68.84 | 91.16 | 83.7 | |
| LoRABackbone=LLaMA-3-8B, #Params=4.72M, Rank (r)=82026.02 | 81.6 | 78.5 | 82.08 | 87.54 | 77.79 | 91.67 | 70.4 | 87.95 | 82.19 | |
| AuroraBackbone=LLaMA-3-8B, #Params=4.72M, Rank (r)=82026.02 | 81.6 | 78.5 | 82.08 | 87.27 | 77.48 | 91.62 | 70.46 | 88.05 | 82.13 | |
| DoRABackbone=LLaMA-3-8B, #Params=4.92M, Rank (r)=82026.02 | 81 | 78.41 | 82.24 | 87.65 | 77.99 | 91.58 | 70.15 | 88.12 | 82.14 | |
| AuroraBackbone=Gemma-7B, #Params=1.20M, Rank (r)=22026.02 | 80 | 81.48 | 79.24 | 85.75 | 75.9 | 90.57 | 68.13 | 85.63 | 80.84 | |
| MELoRABackbone=Gemma-7B, #Params=4.82M, Rank (r)=82026.02 | 79.6 | 79.86 | 78.61 | 84.93 | 74.67 | 89.73 | 69.27 | 87.81 | 80.56 | |
| AuroraBackbone=LLaMA-3-8B, #Params=1.18M, Rank (r)=22026.02 | 76.2 | 77.13 | 72.93 | 84.28 | 73.08 | 89.73 | 67.77 | 82.45 | 77.95 | |
| MELoRABackbone=LLaMA-3-8B, #Params=4.72M, Rank (r)=82026.02 | 74.8 | 75 | 71.43 | 84.71 | 72.98 | 89.44 | 67.74 | 80.84 | 77.12 |