List your company
ResearchFormerly WizWand

Multiple Choice Classification on MMLU

100AccuracyLlama-Guard-3-8B
36.24852.79969.3585.901May 11, 2026May 14, 2026May 17, 2026May 20, 2026May 23, 2026May 26, 2026May 30, 2026
Updated 5mo ago

Evaluation results

MethodLinks
Llama-Guard-3-8Bsize=8B2026.05100
EvoSafety (Guard)-3Bsize=3B2026.0599.47
EvoSafety+ (Guard)-3Bsize=3B, variant=plus2026.0599.47
WildGuard-7Bsize=7B2026.0599.12
Qwen3Guard-8Bsize=8B2026.0599.12
Task CalibrationModel=Gemma-3-12B, Decoding Strategy=Ours2026.0589.72
Task CalibrationModel=Qwen3-30B-A3B, Decoding Strategy=Ours2026.0588.34
Task CalibrationModel=Qwen-3-4B, Decoding Strategy=Ours2026.0583.07
BeamModel=Qwen3-30B-A3B, Decoding Strategy=Beam2026.0580.27
ContrastiveModel=Qwen3-30B-A3B, Decoding Strategy=Contrastive2026.0580.27
AncestralModel=Gemma-3-12B, Decoding Strategy=Ancestral2026.0580.11
GreedyModel=Gemma-3-12B, Decoding Strategy=Greedy2026.0580.11
MBRModel=Gemma-3-12B, Decoding Strategy=MBR2026.0580.11
BeamModel=Gemma-3-12B, Decoding Strategy=Beam2026.0580
ContrastiveModel=Gemma-3-12B, Decoding Strategy=Contrastive2026.0579.44
MBRModel=Qwen3-30B-A3B, Decoding Strategy=MBR2026.0578.48
BeamModel=Qwen-3-4B, Decoding Strategy=Beam2026.0577.92
MBRModel=Qwen-3-4B, Decoding Strategy=MBR2026.0577.92
AncestralModel=Qwen3-30B-A3B, Decoding Strategy=Ancestral2026.0577.91
AncestralModel=Qwen-3-4B, Decoding Strategy=Ancestral2026.0577.47
GreedyModel=Qwen-3-4B, Decoding Strategy=Greedy2026.0577.24
GreedyModel=Qwen3-30B-A3B, Decoding Strategy=Greedy2026.0577.02
ContrastiveModel=Qwen-3-4B, Decoding Strategy=Contrastive2026.0576.68
Task CalibrationModel=Gemma-3-4B, Decoding Strategy=Ours2026.0567.11
UADModel=LLaMA3-8B, Decoding=UAD2026.0565.43
DeLaskModel=LLaMA3-8B, Decoding=DeLask2026.0565.43
RegularModel=LLaMA3-8B, Decoding=Regular2026.0565.25
MBRModel=Gemma-3-4B, Decoding Strategy=MBR2026.0564.56
ITIModel=LLaMA3-8B, Decoding=ITI2026.0564.28
BeamModel=Gemma-3-4B, Decoding Strategy=Beam2026.0562.61
GreedyModel=Gemma-3-4B, Decoding Strategy=Greedy2026.0562.61
AncestralModel=Gemma-3-4B, Decoding Strategy=Ancestral2026.0561.87
ContrastiveModel=Gemma-3-4B, Decoding Strategy=Contrastive2026.0561.64
DOLAModel=LLaMA3-8B, Decoding=DOLA2026.0561.36
ADModel=LLaMA3-8B, Decoding=AD2026.0560.69
QFoRAModel=LLaMA-3.1-8B, Params (M)=28.3, Quantization=4-bit2026.0552.1
QDoRAModel=LLaMA-3.1-8B, Params (M)=57.4, Quantization=4-bit2026.0546.8
ADModel=LLaMA2-7B, Decoding=AD2026.0545.9
ITIModel=LLaMA2-7B, Decoding=ITI2026.0545.85
RegularModel=LLaMA2-7B, Decoding=Regular2026.0545.54
DeLaskModel=LLaMA2-7B, Decoding=DeLask2026.0545.47
UADModel=LLaMA2-7B, Decoding=UAD2026.0545.24
DOLAModel=LLaMA2-7B, Decoding=DOLA2026.0543.57
QLoRAModel=LLaMA-2-7B, Params (M)=56.1, Quantization=4-bit2026.0541
QDoRAModel=LLaMA-2-7B, Params (M)=57.0, Quantization=4-bit2026.0539.9
QFoRAModel=LLaMA-2-7B, Params (M)=28.0, Quantization=4-bit2026.0538.9
QLoRAModel=LLaMA-3.1-8B, Params (M)=56.6, Quantization=4-bit2026.0538.7