Trivia QA on Trivia QA
82.2AccuracyMistral 8x7B
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Mistral 8x7BParameters=8x7B, Shots=5-shot2024.10 | 82.2 | — | |
| Mistral 7BParameters=7B, Shots=5-shot2024.10 | 75.2 | — | |
| Gemma 7BParameters=7B, Shots=5-shot2024.10 | 72.3 | — | |
| Llama 3 8BParameters=8B, Shots=5-shot2024.10 | 67.7 | — | |
| LLMBRACESBase Model=Llama3-8B, Rank (r)=16, Param.=2.1M, Param. (%)=0.03, Zero-shot=true2025.03 | 66.11 | — | |
| LLMBRACESBase Model=Llama3-8B, Rank (r)=32, Param.=4.2M, Param. (%)=0.05, Zero-shot=true2025.03 | 64.56 | — | |
| Phi-3 Mini-4kShots=5-shot2024.10 | 64 | — | |
| Shakti-LLMParameters=2.5B, Shots=5-shot2024.10 | 58.2 | — | |
| LLMBRACESBase Model=Llama2-7B, Rank (r)=16, Param.=2.1M, Param. (%)=0.03, Zero-shot=true2025.03 | 55.2 | — | |
| LLMBRACESBase Model=Llama2-7B, Rank (r)=32, Param.=4.2M, Param. (%)=0.06, Zero-shot=true2025.03 | 52.98 | — | |
| LoRABase Model=Llama2-7B, Rank (r)=16, Param.=8.4M, Param. (%)=0.12, Zero-shot=true2025.03 | 43.93 | — | |
| LoRABase Model=Llama2-7B, Rank (r)=32, Param.=16.8M, Param. (%)=0.25, Zero-shot=true2025.03 | 40.34 | — | |
| LLMBRACESBase Model=Qwen2.5-1.5B, Rank (r)=32, Param.=1.2M, Param. (%)=0.08, Zero-shot=true2025.03 | 36.26 | — | |
| Stable LM 2Params.=1.6B, Tokens=2T2025.06 | 35.6 | — | |
| LLMBRACESBase Model=Qwen2.5-1.5B, Rank (r)=16, Param.=0.6M, Param. (%)=0.04, Zero-shot=true2025.03 | 35.4 | — | |
| LoRABase Model=Llama3-8B, Rank (r)=16, Param.=6.8M, Param. (%)=0.08, Zero-shot=true2025.03 | 34.04 | — | |
| SmolLM 2Params.=1.7B, Tokens=11T2025.06 | 27.1 | — | |
| LoRABase Model=Qwen2.5-1.5B, Rank (r)=16, Param.=2.2M, Param. (%)=0.14, Zero-shot=true2025.03 | 26.47 | — | |
| SmolLMParams.=1.7B, Tokens=1T2025.06 | 25.8 | — | |
| LoRABase Model=Qwen2.5-1.5B, Rank (r)=32, Param.=4.4M, Param. (%)=0.28, Zero-shot=true2025.03 | 24.95 | — | |
| MobileLLAMAParams.=1.4B, Tokens=1.3T2025.06 | 24.5 | — | |
| Qwen 2Params.=1.5B, Tokens=7T2025.06 | 24 | — | |
| AdamParams.=1.4B, Tokens=1T2025.06 | 23.9 | — | |
| TinyLlamaParams.=1.1B, Tokens=2T2025.06 | 23.5 | — | |
| LoRABase Model=Llama3-8B, Rank (r)=32, Param.=13.6M, Param. (%)=0.17, Zero-shot=true2025.03 | 23.15 | — | |
| Muon (OSP)Params.=1.4B, Tokens=1T2025.06 | 22.4 | — | |
| LLAMA 3.2Params.=1.2B2025.06 | 20.7 | — | |
| Qwen 2.5Params.=1.5B2025.06 | 20.6 | — | |
| Qwen 1.5Params.=1.8B, Tokens=2.4T2025.06 | 18.8 | — | |
| OLMoParams.=1.2B, Tokens=3T2025.06 | 17.6 | — | |
| OPTParams.=1.3B, Tokens=0.3T2025.06 | 11.1 | — | |
| PythiaParams.=1.4B, Tokens=0.3T2025.06 | 9.2 | — | |
| BoNModel=Llama 22025.03 | — | 65.8 | |
| BoNModel=Llama 32025.03 | — | 71.8 | |
| BoNModel=Gemma 22025.03 | — | 54.2 | |
| DoLaModel=Llama 22025.03 | — | 64.5 | |
| DoLaModel=Llama 32025.03 | — | 73.6 | |
| DoLaModel=Gemma 22025.03 | — | 46.3 | |
| FSCModel=Llama 22025.03 | — | 63.3 | |
| FSCModel=Llama 32025.03 | — | 74.2 | |
| FSCModel=Gemma 22025.03 | — | 54.8 | |
| GreedyModel=Llama 22025.03 | — | 64.8 | |
| GreedyModel=Llama 32025.03 | — | 72.4 | |
| GreedyModel=Gemma 22025.03 | — | 54 | |
| IDModel=Llama 22025.03 | — | 59.3 | |
| IDModel=Llama 32025.03 | — | 77 | |
| IDModel=Gemma 22025.03 | — | 41.7 | |
| MDModel=Llama 22025.03 | — | 72.1 | |
| MDModel=Llama 32025.03 | — | 80.8 | |
| MDModel=Gemma 22025.03 | — | 64.6 | |
| SRModel=Llama 22025.03 | — | 54.6 | |
| SRModel=Llama 32025.03 | — | 72.7 | |
| SRModel=Gemma 22025.03 | — | 56.3 | |
| USCModel=Llama 22025.03 | — | 66.8 | |
| USCModel=Llama 32025.03 | — | 73.7 | |
| USCModel=Gemma 22025.03 | — | 54.5 |