Question Answering on SocialIQA
83.9AccuracyMixDoRA
Evaluation Results
| Method | Links | |
|---|---|---|
| MixDoRABase Model=LLaMA-2 13B, Trainable Parameters=2.5%2024.04 | 83.9 | |
| MixLoRABase Model=LLaMA-2 13B, Trainable Parameters=2.5%2024.04 | 82.5 | |
| MixDoRABase Model=LLaMA-2 7B, Trainable Parameters=2.9%2024.04 | 80.4 | |
| MixDoRABase Model=LLaMA-3 8B, Trainable Parameters=3.0%2024.04 | 80.1 | |
| DoRABase Model=LLaMA-2 13B, Trainable Parameters=2.4%2024.04 | 80.1 | |
| LoRABase Model=LLaMA-2 13B, Trainable Parameters=2.4%2024.04 | 80 | |
| MixLoRABase Model=LLaMA-3 8B, Trainable Parameters=3.0%2024.04 | 78.8 | |
| LoRABase Model=LLaMA-3 8B, Trainable Parameters=2.6%2024.04 | 78.3 | |
| MixLoRABase Model=LLaMA-2 7B, Trainable Parameters=2.9%2024.04 | 78 | |
| DoRABase Model=LLaMA-3 8B, Trainable Parameters=2.6%2024.04 | 76.2 | |
| MixDoRABase Model=Gemma 2B, Trainable Parameters=4.3%2024.04 | 75.9 | |
| DoRABase Model=LLaMA-2 7B, Trainable Parameters=2.9%2024.04 | 74.1 | |
| MixLoRABase Model=Gemma 2B, Trainable Parameters=4.3%2024.04 | 73.6 | |
| DoRABase Model=Gemma 2B, Trainable Parameters=3.2%2024.04 | 71.9 | |
| LoRABase Model=Gemma 2B, Trainable Parameters=3.2%2024.04 | 71.4 | |
| LoRABase Model=LLaMA-2 7B, Trainable Parameters=2.9%2024.04 | 69.9 | |
| OLMo-1B-0724Model Size Category=XL, Parameters=908M-1.2B, Zero-shot=true, Closed-book=true2025.07 | 49.2 | |
| Ettin-Dec-1BModel Size Category=XL, Parameters=908M-1.2B, Zero-shot=true, Closed-book=true2025.07 | 48.2 | |
| Ettin-Dec-400mModel Size Category=Large, Parameters=360-410M, Zero-shot=true, Closed-book=true2025.07 | 45.5 | |
| Llama-3.2-1BModel Size Category=XL, Parameters=908M-1.2B, Zero-shot=true, Closed-book=true2025.07 | 43.2 | |
| SmolLM2-360mModel Size Category=Large, Parameters=360-410M, Zero-shot=true, Closed-book=true2025.07 | 40.7 | |
| Ettin-Dec-150mModel Size Category=Base, Parameters=135-160M, Zero-shot=true, Closed-book=true2025.07 | 40.1 | |
| SmolLM2-135mModel Size Category=Base, Parameters=135-160M, Zero-shot=true, Closed-book=true2025.07 | 39.4 | |
| Pythia-410mModel Size Category=Large, Parameters=360-410M, Zero-shot=true, Closed-book=true2025.07 | 39 | |
| Ettin-Dec-68mModel Size Category=Small, Parameters=68-82M, Zero-shot=true, Closed-book=true2025.07 | 38.8 | |
| Pythia-160mModel Size Category=Base, Parameters=135-160M, Zero-shot=true, Closed-book=true2025.07 | 36.9 | |
| Ettin-Dec-32mModel Size Category=XS, Parameters=32M, Zero-shot=true, Closed-book=true2025.07 | 36.4 | |
| DistilGPTModel Size Category=Small, Parameters=68-82M, Zero-shot=true, Closed-book=true2025.07 | 36.1 | |
| Ettin-Dec-17mModel Size Category=XXS, Parameters=14-17M, Zero-shot=true, Closed-book=true2025.07 | 35.4 | |
| Pythia-14mModel Size Category=XXS, Parameters=14-17M, Zero-shot=true, Closed-book=true2025.07 | 33.4 |