Question Answering on TruthfulQA (Truthful/Inf Metrics)
88.23Truthful*Inf ScoreLlama2-7B
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Llama2-7BBase Model=Llama2-7B, Watermarking=None2023.11 | 88.23 | 95.1 | 92.78 | — | |
| Vicuna-v1.5-7BBase Model=Vicuna-v1.5-7B, Watermarking=None2023.11 | 81.92 | 93.88 | 87.27 | — | |
| WATME_dictionaryBase Model=Vicuna-v1.5-7B, Watermarking=WATME_dictionary2023.11 | 76.92 | 78.95 | 97.43 | 0.7897 | |
| PROVABLE-MARKBase Model=Vicuna-v1.5-7B, Watermarking=PROVABLE-MARK2023.11 | 71.96 | 74.42 | 96.7 | 0.8796 | |
| WATME_promptingBase Model=Vicuna-v1.5-7B, Watermarking=WATME_prompting2023.11 | 67.87 | 69.65 | 97.45 | 0.7396 | |
| TruthX2024.02 | 65.45 | 72.95 | 89.72 | — | |
| KGW-MARKBase Model=Vicuna-v1.5-7B, Watermarking=KGW-MARK2023.11 | 64.81 | 74.05 | 87.52 | 0.7417 | |
| UNBIASED-MARKBase Model=Vicuna-v1.5-7B, Watermarking=UNBIASED-MARK2023.11 | 62.54 | 70.38 | 88.86 | 0.4855 | |
| WATME_dictionaryBase Model=Llama2-7B, Watermarking=WATME_dictionary2023.11 | 61.14 | 69.28 | 88.25 | 0.8848 | |
| GUMBEL-MARKBase Model=Vicuna-v1.5-7B, Watermarking=GUMBEL-MARK2023.11 | 59.61 | 68.3 | 87.27 | 0.4647 | |
| PROVABLE-MARKBase Model=Llama2-7B, Watermarking=PROVABLE-MARK2023.11 | 58.8 | 64.14 | 91.68 | 0.9555 | |
| TrFrCategory=Representation Editing2024.02 | 54.56 | 67.44 | 80.91 | — | |
| LLaMAModel Size=65B, Prompt Style=QA prompt style (Ouyang et al., 2022)2023.02 | 53 | 57 | — | — | |
| WATME_promptingBase Model=Llama2-7B, Watermarking=WATME_prompting2023.11 | 50.39 | 55.83 | 95.1 | 0.8659 | |
| LLAMA 2Model Size=70B2023.07 | 50.18 | — | — | — | |
| LLAMA 1Model Size=65B2023.07 | 48.71 | — | — | — | |
| KGW-MARKBase Model=Llama2-7B, Watermarking=KGW-MARK2023.11 | 48.2 | 57.16 | 84.33 | 0.8416 | |
| LLaMAModel Size=33B, Prompt Style=QA prompt style (Ouyang et al., 2022)2023.02 | 48 | 52 | — | — | |
| CDCategory=Contrastive Decoding2024.02 | 44.4 | 55.3 | 80.29 | — | |
| LLAMA 1Model Size=33B2023.07 | 44.19 | — | — | — | |
| LLAMA 2Model Size=34B2023.07 | 43.45 | — | — | — | |
| GUMBEL-MARKBase Model=Llama2-7B, Watermarking=GUMBEL-MARK2023.11 | 42.59 | 45.9 | 92.78 | 0.4931 | |
| SH2Category=Contrastive Decoding2024.02 | 42.23 | 64.38 | 65.59 | — | |
| LLAMA 2Model Size=13B2023.07 | 41.86 | — | — | — | |
| LLAMA 1Model Size=13B2023.07 | 41.74 | — | — | — | |
| UNBIASED-MARKBase Model=Llama2-7B, Watermarking=UNBIASED-MARK2023.11 | 41.43 | 44.06 | 93.76 | 0.5051 | |
| DoLaCategory=Contrastive Decoding2024.02 | 41.38 | 42.1 | 98.3 | — | |
| LLaMAModel Size=13B, Prompt Style=QA prompt style (Ouyang et al., 2022)2023.02 | 41 | 47 | — | — | |
| FalconModel Size=40B2023.07 | 40.39 | — | — | — | |
| Supervised Finetuning2024.02 | 36.1 | 47.1 | 76.65 | — | |
| MPTModel Size=30B2023.07 | 35.25 | — | — | — | |
| CSSCategory=Representation Editing2024.02 | 33.4 | 34.7 | 96.25 | — | |
| LLAMA 2Model Size=7B2023.07 | 33.29 | — | — | — | |
| ITICategory=Representation Editing2024.02 | 32.44 | 41.74 | 77.72 | — | |
| Llama-2-7B-Chat2024.02 | 31.9 | 36.96 | 86.29 | — | |
| MPTModel Size=7B2023.07 | 29.13 | — | — | — | |
| LLaMAModel Size=7B, Prompt Style=QA prompt style (Ouyang et al., 2022)2023.02 | 29 | 33 | — | — | |
| LLAMA 1Model Size=7B2023.07 | 27.42 | — | — | — | |
| FalconModel Size=7B2023.07 | 25.95 | — | — | — | |
| GPT-3Model Size=175B, Prompt Style=QA prompt style (Ouyang et al., 2022)2023.02 | 25 | 28 | — | — | |
| GPT-3Model Size=1.3B, Prompt Style=QA prompt style (Ouyang et al., 2022)2023.02 | 19 | 31 | — | — | |
| GPT-3Model Size=6B, Prompt Style=QA prompt style (Ouyang et al., 2022)2023.02 | 19 | 22 | — | — |