Truthfulness Evaluation on TruthfulQA (test)
54.95MC1PromptCD
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| PromptCDModel=QWEN2.5-7B-INSTRUCT2026.02 | 54.95 | 71.13 | — | — | — | 45.93 | — | |
| PromptCDModel=MISTRALV0.3-7B-INSTRUCT2026.02 | 50.79 | 69.58 | — | — | — | 44.24 | — | |
| RECITEModel=MISTRALV0.3-7B-INSTRUCT2026.02 | 49.79 | 68.89 | — | — | — | 39.4 | — | |
| PromptCDModel=LLAMA3-8B-INSTRUCT2026.02 | 49.21 | 69.59 | — | — | — | 43.83 | — | |
| VanillaModel=MISTRALV0.3-7B-INSTRUCT2026.02 | 48.59 | 66.24 | — | — | — | 37.45 | — | |
| RECITEModel=QWEN2.5-7B-INSTRUCT2026.02 | 47.24 | 67.61 | — | — | — | 36.9 | — | |
| VanillaModel=QWEN2.5-7B-INSTRUCT2026.02 | 46.88 | 65.15 | — | — | — | 35.22 | — | |
| SLEDModel=MISTRALV0.3-7B-INSTRUCT2026.02 | 45.41 | 68.43 | — | — | — | 40.35 | — | |
| SLEDModel=QWEN2.5-7B-INSTRUCT2026.02 | 44.92 | 70.35 | — | — | — | 39.82 | — | |
| PromptCDModel=LLAMA2-7B-CHAT2026.02 | 42.59 | 64.05 | — | — | — | 33.89 | — | |
| RECITEModel=LLAMA3-8B-INSTRUCT2026.02 | 41.98 | 61.32 | — | — | — | 33.11 | — | |
| SLEDModel=LLAMA3-8B-INSTRUCT2026.02 | 41.37 | 69.17 | — | — | — | 37.88 | — | |
| DoLAModel=MISTRALV0.3-7B-INSTRUCT2026.02 | 41.12 | 67.99 | — | — | — | 37.51 | — | |
| SEA (N=2000, K=99.8%, L=21)Backbone=LLaMA-2-Chat-7B, N=2000, K=99.8%, L=21, training time (s)=152, inference time (s)=5.932024.05 | 39.41 | 57.15 | 68.05 | 50.67 | 33.66 | — | — | |
| DoLAModel=LLAMA3-8B-INSTRUCT2026.02 | 39.16 | 68.27 | — | — | — | 35.69 | — | |
| SEA (N=1000, K=99%, L=4)Backbone=LLaMA-2-Chat-7B, N=1000, K=99%, L=4, training time (s)=140, inference time (s)=5.082024.05 | 38.31 | 55.27 | 70.38 | 48.96 | 35.25 | — | — | |
| ICD (Prompt-version)Backbone=LLaMA-2-Chat-7B, inference time (s)=9.672024.05 | 37.87 | 57.77 | — | — | — | — | — | |
| SLEDModel=LLAMA2-7B-CHAT2026.02 | 37.08 | 63.83 | — | — | — | 32.92 | — | |
| ITIBackbone=LLaMA-2-Chat-7B, inference time (s)=5.822024.05 | 37.01 | 54.66 | — | — | — | — | — | |
| ICL (LLaMA-2-Chat)Backbone=LLaMA-2-Chat-7B, Strategy=In-Context Learning, inference time (s)=4.92024.05 | 36.96 | 54.68 | 69.4 | 47.36 | 33.29 | — | — | |
| LoRA-FT (LLaMA-2-Chat; N=1000)Backbone=LLaMA-2-Chat-7B, Training samples=1000, training time (s)=299, inference time (s)=5.162024.05 | 35.74 | 54.61 | 91.06 | 48.59 | 42.59 | — | — | |
| DoLAModel=QWEN2.5-7B-INSTRUCT2026.02 | 35.74 | 58.76 | — | — | — | 30.68 | — | |
| LoRA-FT (LLaMA-2-Chat; N=2000)Backbone=LLaMA-2-Chat-7B, Training samples=2000, training time (s)=1190, inference time (s)=5.042024.05 | 35.01 | 54.24 | 92.41 | 47.49 | 42.35 | — | — | |
| VanillaModel=LLAMA2-7B-CHAT2026.02 | 33.66 | 51.29 | — | — | — | 24.91 | — | |
| DOLABackbone=LLaMA-2-Chat-7B, inference time (s)=5.62024.05 | 32.97 | 60.84 | — | — | — | — | — | |
| DoLaModel=LLAMA2-7B-CHAT2026.02 | 32.68 | 61.12 | — | — | — | 29.86 | — | |
| RECITEModel=LLAMA2-7B-CHAT2026.02 | 32.19 | 50.38 | — | — | — | 25.06 | — | |
| VanillaModel=LLAMA3-8B-INSTRUCT2026.02 | 31.95 | 48.83 | — | — | — | 23.92 | — | |
| ICL (LLaMA-2)Backbone=LLaMA-2-7B, Strategy=In-Context Learning2024.05 | 28.39 | 43.42 | — | — | — | — | — | |
| CD (13B-Chat vs. 7B-Chat)Strategy=Contrastive Decoding2024.05 | 28.15 | 54.87 | — | — | — | — | — | |
| BaseBase model=Qwen2.5-1.5B2026.05 | — | — | — | — | — | — | 42.5 | |
| BaseBackbone=Qwen2.5-1.5B2026.05 | — | — | — | — | — | — | 42.5 | |
| CMABase model=Qwen2.5-1.5B2026.05 | — | — | — | — | — | — | 42.1 | |
| CMABackbone=Qwen2.5-1.5B2026.05 | — | — | — | — | — | — | 42.8 | |
| DAREBase model=Qwen2.5-1.5B2026.05 | — | — | — | — | — | — | 39.5 | |
| DAREBackbone=Qwen2.5-1.5B2026.05 | — | — | — | — | — | — | 39.5 | |
| EvoGMBase model=Qwen2.5-1.5B2026.05 | — | — | — | — | — | — | 39.7 | |
| EvoGMBackbone=Qwen2.5-1.5B2026.05 | — | — | — | — | — | — | 44.1 | |
| ICL (LLaMA-2-Chat) w/ Best-of-1Backbone=LLaMA-2-Chat-7B, Distribution=Best-of-12024.05 | — | — | 69.4 | 47.36 | 33.29 | — | — | |
| ICL (LLaMA-2-Chat) w/ Best-of-2Backbone=LLaMA-2-Chat-7B, Distribution=Best-of-22024.05 | — | — | 76.5 | 57.03 | 44.55 | — | — | |
| ICL (LLaMA-2-Chat) w/ Best-of-3Backbone=LLaMA-2-Chat-7B, Distribution=Best-of-32024.05 | — | — | 80.54 | 62.3 | 50.31 | — | — | |
| Model SoupBase model=Qwen2.5-1.5B2026.05 | — | — | — | — | — | — | 28.3 | |
| Model SoupBackbone=Qwen2.5-1.5B2026.05 | — | — | — | — | — | — | 28.3 | |
| Model SwarmBase model=Qwen2.5-1.5B2026.05 | — | — | — | — | — | — | 41 | |
| Model SwarmBackbone=Qwen2.5-1.5B2026.05 | — | — | — | — | — | — | 39.2 | |
| MTLBase model=Qwen2.5-1.5B2026.05 | — | — | — | — | — | — | 30.5 | |
| MTLBackbone=Qwen2.5-1.5B2026.05 | — | — | — | — | — | — | 30.5 | |
| PSO-MergingBase model=Qwen2.5-1.5B2026.05 | — | — | — | — | — | — | 41.8 | |
| PSO-MergingBackbone=Qwen2.5-1.5B2026.05 | — | — | — | — | — | — | 42.1 | |
| SEA w/ Best-of-1Backbone=LLaMA-2-Chat-7B, Distribution=Best-of-1, N=20002024.05 | — | — | 68.05 | 50.67 | 33.66 | — | — | |
| SEA w/ Best-of-2Backbone=LLaMA-2-Chat-7B, Distribution=Best-of-2, N=20002024.05 | — | — | 77.72 | 57.28 | 44.56 | — | — | |
| SEA w/ Best-of-3Backbone=LLaMA-2-Chat-7B, Distribution=Best-of-3, N=20002024.05 | — | — | 82.01 | 63.04 | 51.3 | — | — | |
| Single BestBase model=Qwen2.5-1.5B2026.05 | — | — | — | — | — | — | 38.4 | |
| Single BestBackbone=Qwen2.5-1.5B2026.05 | — | — | — | — | — | — | 38.4 | |
| TABackbone=Qwen2.5-1.5B2026.05 | — | — | — | — | — | — | 38.9 | |
| Task ArithmeticBase model=Qwen2.5-1.5B2026.05 | — | — | — | — | — | — | 38.9 | |
| TIESBase model=Qwen2.5-1.5B2026.05 | — | — | — | — | — | — | 41.6 | |
| TIESBackbone=Qwen2.5-1.5B2026.05 | — | — | — | — | — | — | 41.6 |