Multiple Choice Classification on MMLU
100AccuracyLlama-Guard-3-8B
Evaluation Results
| Method | Links | |
|---|---|---|
| Llama-Guard-3-8Bsize=8B2026.05 | 100 | |
| EvoSafety (Guard)-3Bsize=3B2026.05 | 99.47 | |
| EvoSafety+ (Guard)-3Bsize=3B, variant=plus2026.05 | 99.47 | |
| WildGuard-7Bsize=7B2026.05 | 99.12 | |
| Qwen3Guard-8Bsize=8B2026.05 | 99.12 | |
| Task CalibrationModel=Gemma-3-12B, Decoding Strategy=Ours2026.05 | 89.72 | |
| Task CalibrationModel=Qwen3-30B-A3B, Decoding Strategy=Ours2026.05 | 88.34 | |
| Task CalibrationModel=Qwen-3-4B, Decoding Strategy=Ours2026.05 | 83.07 | |
| BeamModel=Qwen3-30B-A3B, Decoding Strategy=Beam2026.05 | 80.27 | |
| ContrastiveModel=Qwen3-30B-A3B, Decoding Strategy=Contrastive2026.05 | 80.27 | |
| AncestralModel=Gemma-3-12B, Decoding Strategy=Ancestral2026.05 | 80.11 | |
| GreedyModel=Gemma-3-12B, Decoding Strategy=Greedy2026.05 | 80.11 | |
| MBRModel=Gemma-3-12B, Decoding Strategy=MBR2026.05 | 80.11 | |
| BeamModel=Gemma-3-12B, Decoding Strategy=Beam2026.05 | 80 | |
| ContrastiveModel=Gemma-3-12B, Decoding Strategy=Contrastive2026.05 | 79.44 | |
| MBRModel=Qwen3-30B-A3B, Decoding Strategy=MBR2026.05 | 78.48 | |
| BeamModel=Qwen-3-4B, Decoding Strategy=Beam2026.05 | 77.92 | |
| MBRModel=Qwen-3-4B, Decoding Strategy=MBR2026.05 | 77.92 | |
| AncestralModel=Qwen3-30B-A3B, Decoding Strategy=Ancestral2026.05 | 77.91 | |
| AncestralModel=Qwen-3-4B, Decoding Strategy=Ancestral2026.05 | 77.47 | |
| GreedyModel=Qwen-3-4B, Decoding Strategy=Greedy2026.05 | 77.24 | |
| GreedyModel=Qwen3-30B-A3B, Decoding Strategy=Greedy2026.05 | 77.02 | |
| ContrastiveModel=Qwen-3-4B, Decoding Strategy=Contrastive2026.05 | 76.68 | |
| Task CalibrationModel=Gemma-3-4B, Decoding Strategy=Ours2026.05 | 67.11 | |
| UADModel=LLaMA3-8B, Decoding=UAD2026.05 | 65.43 | |
| DeLaskModel=LLaMA3-8B, Decoding=DeLask2026.05 | 65.43 | |
| RegularModel=LLaMA3-8B, Decoding=Regular2026.05 | 65.25 | |
| MBRModel=Gemma-3-4B, Decoding Strategy=MBR2026.05 | 64.56 | |
| ITIModel=LLaMA3-8B, Decoding=ITI2026.05 | 64.28 | |
| BeamModel=Gemma-3-4B, Decoding Strategy=Beam2026.05 | 62.61 | |
| GreedyModel=Gemma-3-4B, Decoding Strategy=Greedy2026.05 | 62.61 | |
| AncestralModel=Gemma-3-4B, Decoding Strategy=Ancestral2026.05 | 61.87 | |
| ContrastiveModel=Gemma-3-4B, Decoding Strategy=Contrastive2026.05 | 61.64 | |
| DOLAModel=LLaMA3-8B, Decoding=DOLA2026.05 | 61.36 | |
| ADModel=LLaMA3-8B, Decoding=AD2026.05 | 60.69 | |
| QFoRAModel=LLaMA-3.1-8B, Params (M)=28.3, Quantization=4-bit2026.05 | 52.1 | |
| QDoRAModel=LLaMA-3.1-8B, Params (M)=57.4, Quantization=4-bit2026.05 | 46.8 | |
| ADModel=LLaMA2-7B, Decoding=AD2026.05 | 45.9 | |
| ITIModel=LLaMA2-7B, Decoding=ITI2026.05 | 45.85 | |
| RegularModel=LLaMA2-7B, Decoding=Regular2026.05 | 45.54 | |
| DeLaskModel=LLaMA2-7B, Decoding=DeLask2026.05 | 45.47 | |
| UADModel=LLaMA2-7B, Decoding=UAD2026.05 | 45.24 | |
| DOLAModel=LLaMA2-7B, Decoding=DOLA2026.05 | 43.57 | |
| QLoRAModel=LLaMA-2-7B, Params (M)=56.1, Quantization=4-bit2026.05 | 41 | |
| QDoRAModel=LLaMA-2-7B, Params (M)=57.0, Quantization=4-bit2026.05 | 39.9 | |
| QFoRAModel=LLaMA-2-7B, Params (M)=28.0, Quantization=4-bit2026.05 | 38.9 | |
| QLoRAModel=LLaMA-3.1-8B, Params (M)=56.6, Quantization=4-bit2026.05 | 38.7 |