List your company
ResearchFormerly WizWand

Question Answering on SocialIQA

83.9AccuracyMixDoRA
31.3845.01558.6572.285Apr 22, 2024Jul 5, 2024Sep 18, 2024Dec 2, 2024Feb 15, 2025May 1, 2025Jul 15, 2025
Updated 7mo ago

Evaluation results

MethodLinks
MixDoRABase Model=LLaMA-2 13B, Trainable Parameters=2.5%2024.0483.9
MixLoRABase Model=LLaMA-2 13B, Trainable Parameters=2.5%2024.0482.5
MixDoRABase Model=LLaMA-2 7B, Trainable Parameters=2.9%2024.0480.4
MixDoRABase Model=LLaMA-3 8B, Trainable Parameters=3.0%2024.0480.1
DoRABase Model=LLaMA-2 13B, Trainable Parameters=2.4%2024.0480.1
LoRABase Model=LLaMA-2 13B, Trainable Parameters=2.4%2024.0480
MixLoRABase Model=LLaMA-3 8B, Trainable Parameters=3.0%2024.0478.8
LoRABase Model=LLaMA-3 8B, Trainable Parameters=2.6%2024.0478.3
MixLoRABase Model=LLaMA-2 7B, Trainable Parameters=2.9%2024.0478
DoRABase Model=LLaMA-3 8B, Trainable Parameters=2.6%2024.0476.2
MixDoRABase Model=Gemma 2B, Trainable Parameters=4.3%2024.0475.9
DoRABase Model=LLaMA-2 7B, Trainable Parameters=2.9%2024.0474.1
MixLoRABase Model=Gemma 2B, Trainable Parameters=4.3%2024.0473.6
DoRABase Model=Gemma 2B, Trainable Parameters=3.2%2024.0471.9
LoRABase Model=Gemma 2B, Trainable Parameters=3.2%2024.0471.4
LoRABase Model=LLaMA-2 7B, Trainable Parameters=2.9%2024.0469.9
OLMo-1B-0724Model Size Category=XL, Parameters=908M-1.2B, Zero-shot=true, Closed-book=true2025.0749.2
Ettin-Dec-1BModel Size Category=XL, Parameters=908M-1.2B, Zero-shot=true, Closed-book=true2025.0748.2
Ettin-Dec-400mModel Size Category=Large, Parameters=360-410M, Zero-shot=true, Closed-book=true2025.0745.5
Llama-3.2-1BModel Size Category=XL, Parameters=908M-1.2B, Zero-shot=true, Closed-book=true2025.0743.2
SmolLM2-360mModel Size Category=Large, Parameters=360-410M, Zero-shot=true, Closed-book=true2025.0740.7
Ettin-Dec-150mModel Size Category=Base, Parameters=135-160M, Zero-shot=true, Closed-book=true2025.0740.1
SmolLM2-135mModel Size Category=Base, Parameters=135-160M, Zero-shot=true, Closed-book=true2025.0739.4
Pythia-410mModel Size Category=Large, Parameters=360-410M, Zero-shot=true, Closed-book=true2025.0739
Ettin-Dec-68mModel Size Category=Small, Parameters=68-82M, Zero-shot=true, Closed-book=true2025.0738.8
Pythia-160mModel Size Category=Base, Parameters=135-160M, Zero-shot=true, Closed-book=true2025.0736.9
Ettin-Dec-32mModel Size Category=XS, Parameters=32M, Zero-shot=true, Closed-book=true2025.0736.4
DistilGPTModel Size Category=Small, Parameters=68-82M, Zero-shot=true, Closed-book=true2025.0736.1
Ettin-Dec-17mModel Size Category=XXS, Parameters=14-17M, Zero-shot=true, Closed-book=true2025.0735.4
Pythia-14mModel Size Category=XXS, Parameters=14-17M, Zero-shot=true, Closed-book=true2025.0733.4