List your company
ResearchFormerly WizWand

Question Answering on TruthfulQA (test)

72AccuracyART-max
13.7628.884459.12Feb 11, 2026Mar 5, 2026Mar 28, 2026Apr 20, 2026May 13, 2026Jun 5, 2026Jun 28, 2026
Updated 4mo ago

Evaluation results

MethodLinks
ART-maxModel=Qwen2.5-32B-Instruct, Evaluation Protocol=D_test2026.0472
ART-meanModel=Qwen2.5-32B-Instruct, Evaluation Protocol=D_test2026.0471.8
VanillaModel=Qwen2.5-32B-Instruct, Evaluation Protocol=D_test2026.0471.6
ART-meanModel=Qwen2.5-14B-Instruct, Evaluation Protocol=D_test2026.0463.3
ART-maxModel=Qwen2.5-14B-Instruct, Evaluation Protocol=D_test2026.0463.1
VanillaModel=Qwen2.5-14B-Instruct, Evaluation Protocol=D_test2026.0462.3
ART-maxModel=Qwen2.5-7B-Instruct, Evaluation Protocol=D_test2026.0454.6
ART-meanModel=Qwen2.5-7B-Instruct, Evaluation Protocol=D_test2026.0454
VanillaModel=Qwen2.5-7B-Instruct, Evaluation Protocol=D_test2026.0451.2
ART-meanModel=Llama3.1-8B-Instruct, Evaluation Protocol=D_test2026.0446.8
VanillaModel=Llama3.1-8B-Instruct, Evaluation Protocol=D_test2026.0446.5
ART-maxModel=Ministral-8B-Instruct-2410, Evaluation Protocol=D_test2026.0446.4
ART-maxModel=Llama3.1-8B-Instruct, Evaluation Protocol=D_test2026.0446.3
ART-meanModel=Ministral-8B-Instruct-2410, Evaluation Protocol=D_test2026.0446.2
VanillaModel=Ministral-8B-Instruct-2410, Evaluation Protocol=D_test2026.0446.1
ART-meanModel=Qwen2-7B-Instruct, Evaluation Protocol=D_test2026.0443.8
ART-maxModel=Qwen2-7B-Instruct, Evaluation Protocol=D_test2026.0443.2
PPT (Distribution)Backbone=LLAMA3-8B2026.0543.2
BaseBackbone=LLAMA3-8B2026.0542.2
VanillaModel=Qwen2-7B-Instruct, Evaluation Protocol=D_test2026.0441.7
PPT (Mean)Backbone=LLAMA3-8B2026.0541.7
BaseBackbone=QWEN2-7B2026.0541.1
PPT (Distribution)Backbone=QWEN2-7B2026.0540.4
Non-adaptive attackModel=Phi-2-2.7B, Attack=Non-adaptive, Defense Status=Before Defense, Quantization=INT82026.0639.5
PPT (Mean)Backbone=QWEN2-7B2026.0538.8
QuantGuardModel=Phi-2-2.7B, Attack=Non-adaptive, Defense Status=After QuantGuard, Quantization=INT82026.0638.2
QuantGuardModel=Phi-2-2.7B, Attack=Adaptive, Defense Status=After QuantGuard, Quantization=INT82026.0635.8
Adaptive attackModel=Phi-2-2.7B, Attack=Adaptive, Defense Status=Before Defense, Quantization=INT82026.0635.5
FoSS2026.0230.45
CoG2026.0229.38
Transformer (w FT)fine-tuned=true, fine-tuning dataset=WikiText-1032026.0228.76
Transformer (w/o FT)fine-tuned=false2026.0228.02
Adaptive attackModel=StarCoder-1B, Attack=Adaptive, Defense Status=Before Defense, Quantization=INT82026.0624.3
Non-adaptive attackModel=StarCoder-1B, Attack=Non-adaptive, Defense Status=Before Defense, Quantization=INT82026.0624
QuantGuardModel=StarCoder-1B, Attack=Non-adaptive, Defense Status=After QuantGuard, Quantization=INT82026.0622.3
ART-maxModel=Llama2-7B-Chat, Evaluation Protocol=D_test2026.0421.7
ART-meanModel=Llama2-7B-Chat, Evaluation Protocol=D_test2026.0421.5
QuantGuardModel=StarCoder-1B, Attack=Adaptive, Defense Status=After QuantGuard, Quantization=INT82026.0620.9
VanillaModel=Llama2-7B-Chat, Evaluation Protocol=D_test2026.0419
VerbalizedBackbone=LLAMA3-8B2026.0516