Question Answering on TruthfulQA (MC1, MC2, MC3, Average Metrics)
86.64AccuracyROAST
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| ROASTModel Size=32B, Backbone=Qwen32026.02 | 86.64 | — | — | — | — | |
| Qwen3Model Size=32B2026.02 | 85.39 | — | — | — | — | |
| ROASTModel Size=14B, Backbone=Qwen32026.02 | 82.32 | — | — | — | — | |
| ROASTModel Size=8B, Backbone=Qwen32026.02 | 81.18 | — | — | — | — | |
| JoBSModel selection method=JoBS, Data selection method=JoBS2026.02 | 80.6 | — | — | — | — | |
| Qwen3Model Size=14B2026.02 | 80.31 | — | — | — | — | |
| Qwen3Model Size=8B2026.02 | 78.66 | — | — | — | — | |
| DARTSModel selection method=DARTS, Data selection method=BO2026.02 | 77 | — | — | — | — | |
| BOModel selection method=BO, Data selection method=BO2026.02 | 76.1 | — | — | — | — | |
| DefaultModel selection method=Default, Data selection method=BO2026.02 | 74.8 | — | — | — | — | |
| RoBoTModel selection method=RoBoT, Data selection method=BO2026.02 | 73.9 | — | — | — | — | |
| AutoLoRAModel selection method=AutoLoRA, Data selection method=BO2026.02 | 72.5 | — | — | — | — | |
| BOModel selection method=BO, Data selection method=IF2026.02 | 70.9 | — | — | — | — | |
| BOModel selection method=BO, Data selection method=DoReMi2026.02 | 69.8 | — | — | — | — | |
| ROASTModel Size=32B, Backbone=GLM42026.02 | 69.51 | — | — | — | — | |
| No SteeringBase Model=Llama 3 8B, Method Category=Baselines, Random Seeds=52025.12 | 69.3 | — | — | — | — | |
| Prompt guardrailsBase Model=Llama 3 8B, Method Category=Baselines, Random Seeds=52025.12 | 69 | — | — | — | — | |
| Gradient CuffBase Model=Llama 3 8B, Method Category=Baselines, Random Seeds=52025.12 | 68.8 | — | — | — | — | |
| Mistral (7B)Implementation=ANN, Bit-channels (Ch.)=322026.01 | 68.26 | — | — | — | — | |
| Mistral (7B)Implementation=SNN (NEXUS), Bit-channels (Ch.)=322026.01 | 68.26 | — | — | — | — | |
| BOModel selection method=BO, Data selection method=LESS2026.02 | 67.9 | — | — | — | — | |
| SAE steeringBase Model=Llama 3 8B, Method Category=Baselines, Random Seeds=52025.12 | 67 | — | — | — | — | |
| AutoLoRAModel selection method=AutoLoRA, Data selection method=LESS2026.02 | 66.9 | — | — | — | — | |
| FULLModel=Mistral-v2-7B, Zero-shot protocol=true2025.12 | 66.82 | — | — | — | — | |
| BOModel selection method=BO, Data selection method=Default2026.02 | 66.8 | — | — | — | — | |
| DARTSModel selection method=DARTS, Data selection method=DoReMi2026.02 | 66.7 | — | — | — | — | |
| GSAE-1DBase Model=Llama 3 8B, Method Category=Ablation, Random Seeds=52025.12 | 66.5 | — | — | — | — | |
| CAABase Model=Llama 3 8B, Method Category=Baselines, Random Seeds=52025.12 | 66.3 | — | — | — | — | |
| AutoLoRAModel selection method=AutoLoRA, Data selection method=Diversity2026.02 | 65.9 | — | — | — | — | |
| RoBoTModel selection method=RoBoT, Data selection method=LESS2026.02 | 65.8 | — | — | — | — | |
| RoBoTModel selection method=RoBoT, Data selection method=Diversity2026.02 | 65.8 | — | — | — | — | |
| GSAEBase Model=Llama 3 8B, Method Category=Main Method, Random Seeds=52025.12 | 65.4 | — | — | — | — | |
| AutoLoRAModel selection method=AutoLoRA, Data selection method=IF2026.02 | 65.3 | — | — | — | — | |
| SafeSwitchBase Model=Llama 3 8B, Method Category=Baselines, Random Seeds=52025.12 | 65.2 | — | — | — | — | |
| DARTSModel selection method=DARTS, Data selection method=Diversity2026.02 | 64.8 | — | — | — | — | |
| BOModel selection method=BO, Data selection method=Diversity2026.02 | 64.7 | — | — | — | — | |
| DefaultModel selection method=Default, Data selection method=Diversity2026.02 | 64.5 | — | — | — | — | |
| AutoLoRAModel selection method=AutoLoRA, Data selection method=DoReMi2026.02 | 64.5 | — | — | — | — | |
| RoBoTModel selection method=RoBoT, Data selection method=Default2026.02 | 64.2 | — | — | — | — | |
| DARTSModel selection method=DARTS, Data selection method=IF2026.02 | 63.8 | — | — | — | — | |
| Input Gate OnlyBase Model=Llama 3 8B, Method Category=Ablation, Random Seeds=52025.12 | 63.3 | — | — | — | — | |
| DefaultModel selection method=Default, Data selection method=DoReMi2026.02 | 62.9 | — | — | — | — | |
| DARTSModel selection method=DARTS, Data selection method=LESS2026.02 | 62.4 | — | — | — | — | |
| DefaultModel selection method=Default, Data selection method=IF2026.02 | 62.3 | — | — | — | — | |
| DARTSModel selection method=DARTS, Data selection method=Default2026.02 | 61.9 | — | — | — | — | |
| AutoLoRAModel selection method=AutoLoRA, Data selection method=Default2026.02 | 61.7 | — | — | — | — | |
| DefaultModel selection method=Default, Data selection method=LESS2026.02 | 60.7 | — | — | — | — | |
| No gatingBase Model=Llama 3 8B, Method Category=Ablation, Random Seeds=52025.12 | 60.3 | — | — | — | — | |
| ROASTModel Size=9B, Backbone=GLM42026.02 | 60.1 | — | — | — | — | |
| DefaultModel selection method=Default, Data selection method=Default2026.02 | 59.8 | — | — | — | — | |
| RoBoTModel selection method=RoBoT, Data selection method=DoReMi2026.02 | 58.9 | — | — | — | — | |
| GLM4Model Size=32B2026.02 | 57.4 | — | — | — | — | |
| SPONModel=Mistral-v2-7B, Zero-shot protocol=true2025.12 | 57.15 | — | — | — | — | |
| TEALModel=Mistral-v2-7B, Zero-shot protocol=true2025.12 | 57.08 | — | — | — | — | |
| RoBoTModel selection method=RoBoT, Data selection method=IF2026.02 | 56.9 | — | — | — | — | |
| FULLModel=Qwen3-8b, Zero-shot protocol=true2025.12 | 54.41 | — | — | — | — | |
| Random graphsBase Model=Llama 3 8B, Method Category=Ablation, Random Seeds=52025.12 | 54.2 | — | — | — | — | |
| FULLModel=Llama3-8b, Zero-shot protocol=true2025.12 | 54.03 | — | — | — | — | |
| GLM4Model Size=9B2026.02 | 53.83 | — | — | — | — | |
| SPONModel=Qwen3-8b, Zero-shot protocol=true2025.12 | 53.59 | — | — | — | — | |
| TEALModel=Qwen3-8b, Zero-shot protocol=true2025.12 | 52.78 | — | — | — | — | |
| ROASTModel Size=0.6B, Backbone=Qwen32026.02 | 49.22 | — | — | — | — | |
| Qwen3Model Size=0.6B2026.02 | 48.78 | — | — | — | — | |
| Llama-2 (70B)Implementation=ANN, Bit-channels (Ch.)=322026.01 | 44.9 | — | — | — | — | |
| Llama-2 (70B)Implementation=SNN (NEXUS), Bit-channels (Ch.)=322026.01 | 44.9 | — | — | — | — | |
| Phi-2 (2.7B)Implementation=ANN, Bit-channels (Ch.)=322026.01 | 44.47 | — | — | — | — | |
| Phi-2 (2.7B)Implementation=SNN (NEXUS), Bit-channels (Ch.)=322026.01 | 44.47 | — | — | — | — | |
| SPONModel=Llama3-8b, Zero-shot protocol=true2025.12 | 41.69 | — | — | — | — | |
| Llama-2 (7B)Implementation=ANN, Bit-channels (Ch.)=322026.01 | 40.95 | — | — | — | — | |
| Llama-2 (7B)Implementation=SNN (NEXUS), Bit-channels (Ch.)=322026.01 | 40.95 | — | — | — | — | |
| TEALModel=Llama3-8b, Zero-shot protocol=true2025.12 | 40.14 | — | — | — | — | |
| Qwen3-0.6BImplementation=ANN, Bit-channels (Ch.)=322026.01 | 38.9 | — | — | — | — | |
| Qwen3-0.6BImplementation=SNN (NEXUS), Bit-channels (Ch.)=322026.01 | 38.9 | — | — | — | — | |
| CDModel Used=13B-Chat vs. 7B-Chat2026.01 | — | 28.2 | 54.9 | 29.8 | 37.6 | |
| CDModel Used=70B-Chat vs. 7B-Chat2026.01 | — | 33.7 | 60 | 33 | 42.2 | |
| DHIModel Used=7B-Chat vs. 7B-Finetuned2026.01 | — | 41.9 | 72.6 | 45 | 53.2 | |
| DoLaModel Used=7B-Chat2026.01 | — | 33 | 60.8 | 29.5 | 41.1 | |
| Greedy (Baseline)Model Used=7B-Base2026.01 | — | 28.7 | 43.3 | 20.8 | 30.9 | |
| Greedy (Baseline)Model Used=7B-Chat2026.01 | — | 37.6 | 54.6 | 28.1 | 40.1 | |
| Greedy (Baseline)Model Used=70B-Chat2026.01 | — | 37.7 | 59 | 29.8 | 42.2 | |
| ICDModel Used=7B-Chat vs. 7B-Finetuned2026.01 | — | 40.5 | 69.7 | 41.3 | 50.5 | |
| ITIModel Used=7B-Chat2026.01 | — | 37 | 54.7 | 27.8 | 39.8 |