Truthful QA
68.4AccuracyShakti-LLM
Evaluation Results
| Method | Links | |
|---|---|---|
| Shakti-LLMParameters=2.5B, Shots=10-shot2024.10 | 68.4 | |
| PRIORITYRULESBackbone=Llama-2-13B, Training Data=ShareGPT, Reference Signal (+Lref)=true2024.02 | 66.7 | |
| Phi-3 Mini-4kShots=10-shot2024.10 | 65 | |
| PRIORITYRULESBackbone=Llama-2-13B, Data Censorship=Uncensored, Reference Signal (+Lref)=true2024.02 | 64.5 | |
| MAT-STEERBackbone=Qwen2.5-7B2025.02 | 64.36 | |
| Llama 3 8BParameters=8B, Shots=10-shot2024.10 | 63.1 | |
| LITOBackbone=Qwen2.5-7B2025.02 | 62.37 | |
| NL-ITIBackbone=Qwen2.5-7B2025.02 | 61.45 | |
| PRIORITYRULESBackbone=Llama-2-13B, Training Data=ShareGPT2024.02 | 60.4 | |
| Mistral 8x7BParameters=8x7B, Shots=10-shot2024.10 | 60.1 | |
| ICVBackbone=Qwen2.5-7B2025.02 | 59.94 | |
| Llama-2 ChatBackbone=Llama-2-13B2024.02 | 59.8 | |
| Qwen1.5-72B2024.07 | 59.6 | |
| RCOBase Model=LLaMA-3-8B-Instruct2025.06 | 59.58 | |
| DPOBackbone=Qwen2.5-7B2025.02 | 59.56 | |
| LLaMA-3-70B-Instruct2025.06 | 58.21 | |
| ITIBackbone=Qwen2.5-7B2025.02 | 58.09 | |
| PRIORITYRULESBackbone=Llama-2-7B, Data Censorship=Uncensored, Reference Signal (+Lref)=true2024.02 | 57.7 | |
| Qwen2-57B-A14BArchitecture=MoE, # Act Params=14B, # Params=57B2024.07 | 57.7 | |
| RAdaptBackbone=Qwen2.5-7B2025.02 | 57.5 | |
| Qwen1.5-32BArchitecture=Dense, # Act Params=34B, # Params=34B2024.07 | 57.4 | |
| MergeBackbone=Qwen2.5-7B2025.02 | 57.16 | |
| ICLBackbone=Qwen2.5-7B2025.02 | 57.12 | |
| PRIORITYRULESBackbone=Llama-2-13B, Data Censorship=Uncensored2024.02 | 56.9 | |
| Llama-2 ChatBackbone=Llama-2-7B2024.02 | 56.6 | |
| ShareGPTBackbone=Llama-2-13B2024.02 | 56.6 | |
| SFTBackbone=Qwen2.5-7B2025.02 | 56.52 | |
| RCOBase Model=Auto-J-13B2025.06 | 56.19 | |
| PRIORITYRULESBackbone=Llama-2-7B, Data Censorship=Uncensored2024.02 | 56.1 | |
| UltraCM-13B2025.06 | 55.98 | |
| ChatUse Sens=false, Backbone=Mistral-7B2025.02 | 55.69 | |
| RCOBase Model=UltraCM-13B2025.06 | 55.62 | |
| RCOBase Model=LLaMA-2-13B-Chat2025.06 | 55.31 | |
| RCOBase Model=LLaMA-2-7B-Chat2025.06 | 54.96 | |
| Qwen2-72B2024.07 | 54.8 | |
| DPCOBase Model=Auto-J-13B2025.06 | 54.72 | |
| LLaMA-3-8B-Instruct2025.06 | 54.32 | |
| DPCOBase Model=UltraCM-13B2025.06 | 54.31 | |
| Qwen2.5-7BBackbone=Qwen2.5-7B, Status=Base Model2025.02 | 54.06 | |
| Yi-1.5-34BArchitecture=Dense, # Act Params=32B, # Params=32B2024.07 | 53.9 | |
| Mistral 7BParameters=7B, Shots=10-shot2024.10 | 53 | |
| Self-refinement2025.06 | 52.39 | |
| PRIORITYRULESBackbone=Llama-2-7B, Training Data=ShareGPT2024.02 | 52.2 | |
| Gemma 7BParameters=7B, Shots=10-shot2024.10 | 52.1 | |
| PRIORITYRULESBackbone=Llama-2-7B, Training Data=ShareGPT, Reference Signal (+Lref)=true2024.02 | 51.8 | |
| Mixtral-8x7BArchitecture=MoE, # Act Params=12B, # Params=47B2024.07 | 51.1 | |
| Mixtral-8x22B2024.07 | 51 | |
| DPCOBase Model=LLaMA-3-8B-Instruct2025.06 | 50.7 | |
| DPCOBase Model=LLaMA-2-7B-Chat2025.06 | 50.43 | |
| Auto-J-13B2025.06 | 50.4 | |
| DPCOBase Model=LLaMA-2-13B-Chat2025.06 | 50.28 | |
| LLAMA-2-70B-Chat2025.06 | 50.23 | |
| LLaMA-2-13B-Chat2025.06 | 49.81 | |
| Qwen1.5-110B2024.07 | 49.6 | |
| Initial Answer2025.06 | 49.12 | |
| PRIORITYRULESBackbone=Llama-2-13B, System Message (+sys)=true, Training Data=ShareGPT2024.02 | 49.1 | |
| PRIORITYRULESBackbone=Llama-2-13B, Data Censorship=Uncensored, System Message (+sys)=true2024.02 | 49.1 | |
| Unc. ShareGPTBackbone=Llama-2-13B, Data Censorship=Uncensored2024.02 | 49.1 | |
| Ties-MergingUse Sens=false, Backbone=Mistral-7B2025.02 | 48.71 | |
| LLaMA-2-7B-Chat2025.06 | 48.32 | |
| Ties-MergingUse Sens=true, Backbone=Mistral-7B2025.02 | 48.12 | |
| DAREUse Sens=true, Backbone=Mistral-7B2025.02 | 46.88 | |
| Aligner2025.06 | 46.61 | |
| JambaArchitecture=MoE, # Act Params=12B, # Params=52B2024.07 | 46.4 | |
| ShareGPTBackbone=Llama-2-7B2024.02 | 46.3 | |
| Unc. ShareGPTBackbone=Llama-2-7B, Data Censorship=Uncensored2024.02 | 46 | |
| Qwen2-1.5B# Non-Emb Params=1.2B2024.07 | 45.9 | |
| Llama-3-70B2024.07 | 45.6 | |
| CodeUse Sens=false, Backbone=Mistral-7B2025.02 | 45.29 | |
| Task ArithmeticUse Sens=true, Backbone=Mistral-7B2025.02 | 45.29 | |
| ShareGPT (+sys)Backbone=Llama-2-13B, System Message (+sys)=true2024.02 | 44.7 | |
| MathUse Sens=false, Backbone=Mistral-7B2025.02 | 44.68 | |
| Phi-2# Non-Emb Params=2.5B2024.07 | 44.5 | |
| DAREUse Sens=false, Backbone=Mistral-7B2025.02 | 42.84 | |
| Unc. ShareGPT (+sys)Backbone=Llama-2-13B, Data Censorship=Uncensored, System Message (+sys)=true2024.02 | 41.2 | |
| Task ArithmeticUse Sens=false, Backbone=Mistral-7B2025.02 | 41 | |
| Qwen2-0.5B# Non-Emb Params=0.3B2024.07 | 39.7 | |
| Qwen1.5-1.8B# Non-Emb Params=1.2B2024.07 | 39.4 | |
| PRIORITYRULESBackbone=Llama-2-7B, Data Censorship=Uncensored, System Message (+sys)=true2024.02 | 37.9 | |
| PRIORITYRULESBackbone=Llama-2-7B, System Message (+sys)=true, Training Data=ShareGPT2024.02 | 37.4 | |
| Unc. ShareGPT (+sys)Backbone=Llama-2-7B, Data Censorship=Uncensored, System Message (+sys)=true2024.02 | 34.9 | |
| Gemma-2B# Non-Emb Params=2.0B2024.07 | 33.1 | |
| ShareGPT (+sys)Backbone=Llama-2-7B, System Message (+sys)=true2024.02 | 32.3 |