List your company
ResearchFormerly WizWand

Truthful QA

68.4AccuracyShakti-LLM
30.85640.60350.3560.097Feb 27, 2024May 18, 2024Aug 7, 2024Oct 27, 2024Jan 16, 2025Apr 7, 2025Jun 27, 2025
Updated 8mo ago

Evaluation results

MethodLinks
Shakti-LLMParameters=2.5B, Shots=10-shot2024.1068.4
PRIORITYRULESBackbone=Llama-2-13B, Training Data=ShareGPT, Reference Signal (+Lref)=true2024.0266.7
Phi-3 Mini-4kShots=10-shot2024.1065
PRIORITYRULESBackbone=Llama-2-13B, Data Censorship=Uncensored, Reference Signal (+Lref)=true2024.0264.5
MAT-STEERBackbone=Qwen2.5-7B2025.0264.36
Llama 3 8BParameters=8B, Shots=10-shot2024.1063.1
LITOBackbone=Qwen2.5-7B2025.0262.37
NL-ITIBackbone=Qwen2.5-7B2025.0261.45
PRIORITYRULESBackbone=Llama-2-13B, Training Data=ShareGPT2024.0260.4
Mistral 8x7BParameters=8x7B, Shots=10-shot2024.1060.1
ICVBackbone=Qwen2.5-7B2025.0259.94
Llama-2 ChatBackbone=Llama-2-13B2024.0259.8
Qwen1.5-72B2024.0759.6
RCOBase Model=LLaMA-3-8B-Instruct2025.0659.58
DPOBackbone=Qwen2.5-7B2025.0259.56
LLaMA-3-70B-Instruct2025.0658.21
ITIBackbone=Qwen2.5-7B2025.0258.09
PRIORITYRULESBackbone=Llama-2-7B, Data Censorship=Uncensored, Reference Signal (+Lref)=true2024.0257.7
Qwen2-57B-A14BArchitecture=MoE, # Act Params=14B, # Params=57B2024.0757.7
RAdaptBackbone=Qwen2.5-7B2025.0257.5
Qwen1.5-32BArchitecture=Dense, # Act Params=34B, # Params=34B2024.0757.4
MergeBackbone=Qwen2.5-7B2025.0257.16
ICLBackbone=Qwen2.5-7B2025.0257.12
PRIORITYRULESBackbone=Llama-2-13B, Data Censorship=Uncensored2024.0256.9
Llama-2 ChatBackbone=Llama-2-7B2024.0256.6
ShareGPTBackbone=Llama-2-13B2024.0256.6
SFTBackbone=Qwen2.5-7B2025.0256.52
RCOBase Model=Auto-J-13B2025.0656.19
PRIORITYRULESBackbone=Llama-2-7B, Data Censorship=Uncensored2024.0256.1
UltraCM-13B2025.0655.98
ChatUse Sens=false, Backbone=Mistral-7B2025.0255.69
RCOBase Model=UltraCM-13B2025.0655.62
RCOBase Model=LLaMA-2-13B-Chat2025.0655.31
RCOBase Model=LLaMA-2-7B-Chat2025.0654.96
Qwen2-72B2024.0754.8
DPCOBase Model=Auto-J-13B2025.0654.72
LLaMA-3-8B-Instruct2025.0654.32
DPCOBase Model=UltraCM-13B2025.0654.31
Qwen2.5-7BBackbone=Qwen2.5-7B, Status=Base Model2025.0254.06
Yi-1.5-34BArchitecture=Dense, # Act Params=32B, # Params=32B2024.0753.9
Mistral 7BParameters=7B, Shots=10-shot2024.1053
Self-refinement2025.0652.39
PRIORITYRULESBackbone=Llama-2-7B, Training Data=ShareGPT2024.0252.2
Gemma 7BParameters=7B, Shots=10-shot2024.1052.1
PRIORITYRULESBackbone=Llama-2-7B, Training Data=ShareGPT, Reference Signal (+Lref)=true2024.0251.8
Mixtral-8x7BArchitecture=MoE, # Act Params=12B, # Params=47B2024.0751.1
Mixtral-8x22B2024.0751
DPCOBase Model=LLaMA-3-8B-Instruct2025.0650.7
DPCOBase Model=LLaMA-2-7B-Chat2025.0650.43
Auto-J-13B2025.0650.4
DPCOBase Model=LLaMA-2-13B-Chat2025.0650.28
LLAMA-2-70B-Chat2025.0650.23
LLaMA-2-13B-Chat2025.0649.81
Qwen1.5-110B2024.0749.6
Initial Answer2025.0649.12
PRIORITYRULESBackbone=Llama-2-13B, System Message (+sys)=true, Training Data=ShareGPT2024.0249.1
PRIORITYRULESBackbone=Llama-2-13B, Data Censorship=Uncensored, System Message (+sys)=true2024.0249.1
Unc. ShareGPTBackbone=Llama-2-13B, Data Censorship=Uncensored2024.0249.1
Ties-MergingUse Sens=false, Backbone=Mistral-7B2025.0248.71
LLaMA-2-7B-Chat2025.0648.32
Ties-MergingUse Sens=true, Backbone=Mistral-7B2025.0248.12
DAREUse Sens=true, Backbone=Mistral-7B2025.0246.88
Aligner2025.0646.61
JambaArchitecture=MoE, # Act Params=12B, # Params=52B2024.0746.4
ShareGPTBackbone=Llama-2-7B2024.0246.3
Unc. ShareGPTBackbone=Llama-2-7B, Data Censorship=Uncensored2024.0246
Qwen2-1.5B# Non-Emb Params=1.2B2024.0745.9
Llama-3-70B2024.0745.6
CodeUse Sens=false, Backbone=Mistral-7B2025.0245.29
Task ArithmeticUse Sens=true, Backbone=Mistral-7B2025.0245.29
ShareGPT (+sys)Backbone=Llama-2-13B, System Message (+sys)=true2024.0244.7
MathUse Sens=false, Backbone=Mistral-7B2025.0244.68
Phi-2# Non-Emb Params=2.5B2024.0744.5
DAREUse Sens=false, Backbone=Mistral-7B2025.0242.84
Unc. ShareGPT (+sys)Backbone=Llama-2-13B, Data Censorship=Uncensored, System Message (+sys)=true2024.0241.2
Task ArithmeticUse Sens=false, Backbone=Mistral-7B2025.0241
Qwen2-0.5B# Non-Emb Params=0.3B2024.0739.7
Qwen1.5-1.8B# Non-Emb Params=1.2B2024.0739.4
PRIORITYRULESBackbone=Llama-2-7B, Data Censorship=Uncensored, System Message (+sys)=true2024.0237.9
PRIORITYRULESBackbone=Llama-2-7B, System Message (+sys)=true, Training Data=ShareGPT2024.0237.4
Unc. ShareGPT (+sys)Backbone=Llama-2-7B, Data Censorship=Uncensored, System Message (+sys)=true2024.0234.9
Gemma-2B# Non-Emb Params=2.0B2024.0733.1
ShareGPT (+sys)Backbone=Llama-2-7B, System Message (+sys)=true2024.0232.3