Question Answering on TruthfulQA (test)
72AccuracyART-max
Evaluation Results
| Method | Links | |
|---|---|---|
| ART-maxModel=Qwen2.5-32B-Instruct, Evaluation Protocol=D_test2026.04 | 72 | |
| ART-meanModel=Qwen2.5-32B-Instruct, Evaluation Protocol=D_test2026.04 | 71.8 | |
| VanillaModel=Qwen2.5-32B-Instruct, Evaluation Protocol=D_test2026.04 | 71.6 | |
| ART-meanModel=Qwen2.5-14B-Instruct, Evaluation Protocol=D_test2026.04 | 63.3 | |
| ART-maxModel=Qwen2.5-14B-Instruct, Evaluation Protocol=D_test2026.04 | 63.1 | |
| VanillaModel=Qwen2.5-14B-Instruct, Evaluation Protocol=D_test2026.04 | 62.3 | |
| ART-maxModel=Qwen2.5-7B-Instruct, Evaluation Protocol=D_test2026.04 | 54.6 | |
| ART-meanModel=Qwen2.5-7B-Instruct, Evaluation Protocol=D_test2026.04 | 54 | |
| VanillaModel=Qwen2.5-7B-Instruct, Evaluation Protocol=D_test2026.04 | 51.2 | |
| ART-meanModel=Llama3.1-8B-Instruct, Evaluation Protocol=D_test2026.04 | 46.8 | |
| VanillaModel=Llama3.1-8B-Instruct, Evaluation Protocol=D_test2026.04 | 46.5 | |
| ART-maxModel=Ministral-8B-Instruct-2410, Evaluation Protocol=D_test2026.04 | 46.4 | |
| ART-maxModel=Llama3.1-8B-Instruct, Evaluation Protocol=D_test2026.04 | 46.3 | |
| ART-meanModel=Ministral-8B-Instruct-2410, Evaluation Protocol=D_test2026.04 | 46.2 | |
| VanillaModel=Ministral-8B-Instruct-2410, Evaluation Protocol=D_test2026.04 | 46.1 | |
| ART-meanModel=Qwen2-7B-Instruct, Evaluation Protocol=D_test2026.04 | 43.8 | |
| ART-maxModel=Qwen2-7B-Instruct, Evaluation Protocol=D_test2026.04 | 43.2 | |
| PPT (Distribution)Backbone=LLAMA3-8B2026.05 | 43.2 | |
| BaseBackbone=LLAMA3-8B2026.05 | 42.2 | |
| VanillaModel=Qwen2-7B-Instruct, Evaluation Protocol=D_test2026.04 | 41.7 | |
| PPT (Mean)Backbone=LLAMA3-8B2026.05 | 41.7 | |
| BaseBackbone=QWEN2-7B2026.05 | 41.1 | |
| PPT (Distribution)Backbone=QWEN2-7B2026.05 | 40.4 | |
| Non-adaptive attackModel=Phi-2-2.7B, Attack=Non-adaptive, Defense Status=Before Defense, Quantization=INT82026.06 | 39.5 | |
| PPT (Mean)Backbone=QWEN2-7B2026.05 | 38.8 | |
| QuantGuardModel=Phi-2-2.7B, Attack=Non-adaptive, Defense Status=After QuantGuard, Quantization=INT82026.06 | 38.2 | |
| QuantGuardModel=Phi-2-2.7B, Attack=Adaptive, Defense Status=After QuantGuard, Quantization=INT82026.06 | 35.8 | |
| Adaptive attackModel=Phi-2-2.7B, Attack=Adaptive, Defense Status=Before Defense, Quantization=INT82026.06 | 35.5 | |
| FoSS2026.02 | 30.45 | |
| CoG2026.02 | 29.38 | |
| Transformer (w FT)fine-tuned=true, fine-tuning dataset=WikiText-1032026.02 | 28.76 | |
| Transformer (w/o FT)fine-tuned=false2026.02 | 28.02 | |
| Adaptive attackModel=StarCoder-1B, Attack=Adaptive, Defense Status=Before Defense, Quantization=INT82026.06 | 24.3 | |
| Non-adaptive attackModel=StarCoder-1B, Attack=Non-adaptive, Defense Status=Before Defense, Quantization=INT82026.06 | 24 | |
| QuantGuardModel=StarCoder-1B, Attack=Non-adaptive, Defense Status=After QuantGuard, Quantization=INT82026.06 | 22.3 | |
| ART-maxModel=Llama2-7B-Chat, Evaluation Protocol=D_test2026.04 | 21.7 | |
| ART-meanModel=Llama2-7B-Chat, Evaluation Protocol=D_test2026.04 | 21.5 | |
| QuantGuardModel=StarCoder-1B, Attack=Adaptive, Defense Status=After QuantGuard, Quantization=INT82026.06 | 20.9 | |
| VanillaModel=Llama2-7B-Chat, Evaluation Protocol=D_test2026.04 | 19 | |
| VerbalizedBackbone=LLAMA3-8B2026.05 | 16 |