Commonsense Question Answering on ARC-E
89.88AccuracyLoRA-Mixer(ours)
Evaluation results
| Method | Links | |
|---|---|---|
| LoRA-Mixer(ours)Base Model=LLaMA-3 (8B)2025.06 | 89.88 | |
| LoRABase Model=LLaMA-3 (8B)2025.06 | 89.59 | |
| MixLoRABase Model=LLaMA-3 (8B)2025.06 | 88.7 | |
| MOLEBase Model=LLaMA-3 (8B)2025.06 | 88.15 | |
| LoRA-Mixer (ours)Base Model=Mistral (7B)2025.06 | 87.87 | |
| LoRAHubBase Model=LLaMA-3 (8B)2025.06 | 87.13 | |
| LoRA-Mixer (ours)Base Model=Falcon-Mamba (7B)2025.06 | 86.87 | |
| LoRABase Model=Mistral (7B)2025.06 | 86.66 | |
| LoRABase Model=Falcon-Mamba (7B)2025.06 | 85.68 | |
| MOLEBase Model=Mistral (7B)2025.06 | 85.68 | |
| MixLoRABase Model=Mistral (7B)2025.06 | 85.42 | |
| MOLEBase Model=Falcon-Mamba (7B)2025.06 | 83.46 | |
| LoRAHubBase Model=Falcon-Mamba (7B)2025.06 | 81.16 | |
| LoRAHubBase Model=Mistral (7B)2025.06 | 81.14 | |
| DenseTraining Tokens=20B, Inference Top-K=N/A2026.01 | 72.31 | |
| DenseTraining Tokens=20B, Inference Top-K=22026.01 | 72.22 | |
| PHSATraining Tokens=20B, Inference Top-K=N/A2026.01 | 72.2 | |
| PHSATraining Tokens=20B, Inference Top-K=22026.01 | 71.97 | |
| InfLLM v2Training Tokens=20B, Inference Top-K=N/A2026.01 | 71.89 | |
| InfLLM v2Training Tokens=20B, Inference Top-K=22026.01 | 71.51 | |
| Qwen-0.6B-BaseTraining Tokens=N/A, Inference Top-K=22026.01 | 65.74 | |
| Qwen-0.6B-BaseTraining Tokens=N/A, Inference Top-K=N/A2026.01 | 65.66 | |
| FP16Bits=-, Backbone=LLAMA-2-7B, Zero-shot=true2026.04 | 53.58 | |
| CBQBits=W4A4, Backbone=LLAMA-2-7B, Zero-shot=true2026.04 | 46.23 | |
| BWABits=W(1+1)A4, Backbone=LLAMA-2-7B, Zero-shot=true2026.04 | 46.13 | |
| LBLLMBits=W(1+1)A4, Backbone=LLAMA-2-7B, Zero-shot=true2026.04 | 44.65 | |
| ARB-LLMBits=W(1+1)A4, Backbone=LLAMA-2-7B, Zero-shot=true2026.04 | 39.31 | |
| QuaRotBits=W2A4, Backbone=LLAMA-2-7B, Zero-shot=true2026.04 | 28.45 | |
| BiLLMBits=W(1+1)A4, Backbone=LLAMA-2-7B, Zero-shot=true2026.04 | 25.38 |