Question Answering on TruthfulQA zero-shot
62.03TruthfulQA ScoreSolar-0-70B
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Solar-0-70BModel=Solar-0-70B, Mode=Baseline2024.02 | 62.03 | 73.77 | |
| BitDeltaModel=Solar-0-70B2024.02 | 62.03 | 73.57 | |
| BitDelta-InitialModel=Solar-0-70B2024.02 | 59.08 | 73.14 | |
| BitDeltaModel=Zephyr-7B-β2024.02 | 58.39 | 66.2 | |
| Mistral-7B v0.1 InstructModel=Mistral-7B v0.1 Instruct, Mode=Baseline2024.02 | 55.93 | 60.36 | |
| BitDeltaModel=Mistral-7B v0.1 Instruct2024.02 | 55.23 | 61.1 | |
| Zephyr-7B-βModel=Zephyr-7B-β, Mode=Baseline2024.02 | 55.12 | 65.22 | |
| Instella-3B-SFT (UM-190k)DPO Training Dataset=UM-190k, Evaluation Protocol=0-shot2025.11 | 54.93 | — | |
| BitDeltaModel=Dolphin 2.2.12024.02 | 54.91 | 66.97 | |
| Instella-3B-SFT (UM-187k)DPO Training Dataset=UM-187k, Evaluation Protocol=0-shot2025.11 | 54.91 | — | |
| BitDelta-InitialModel=Zephyr-7B-β2024.02 | 54.53 | 66.12 | |
| Instella-3B-SFT (UM-170k)DPO Training Dataset=UM-170k, Evaluation Protocol=0-shot2025.11 | 54.45 | — | |
| Instella-3B-SFT (TuluDPO)DPO Training Dataset=TuluDPO, Evaluation Protocol=0-shot2025.11 | 54.21 | — | |
| Dolphin 2.2.1Model=Dolphin 2.2.1, Mode=Baseline2024.02 | 54.02 | 67.31 | |
| Instella-3B-SFT (ORPO)DPO Training Dataset=ORPO, Evaluation Protocol=0-shot2025.11 | 53.35 | — | |
| Llama 2-70B ChatModel=Llama 2-70B Chat, Mode=Baseline2024.02 | 52.77 | 68.82 | |
| Instella-3B-SFT (HelpSteer)DPO Training Dataset=HelpSteer, Evaluation Protocol=0-shot2025.11 | 52.7 | — | |
| Instella-3B-SFT (UltraFB)DPO Training Dataset=UltraFB, Evaluation Protocol=0-shot2025.11 | 52.67 | — | |
| Instella-3B-SFT (SFT)DPO Training Dataset=None (SFT Baseline), Evaluation Protocol=0-shot2025.11 | 52.48 | — | |
| Instella-3B-SFT (CodePref)DPO Training Dataset=CodePref, Evaluation Protocol=0-shot2025.11 | 52.19 | — | |
| BitDeltaModel=Llama 2-70B Chat2024.02 | 51.37 | 69.14 | |
| BitDelta-InitialModel=Mistral-7B v0.1 Instruct2024.02 | 51.27 | 63.83 | |
| BitDelta-InitialModel=Dolphin 2.2.12024.02 | 48.14 | 67.58 | |
| Llama 2-70BModel=Llama 2-70B2024.02 | 44.82 | 71.81 | |
| Mistral-7B v0.1Model=Mistral-7B v0.12024.02 | 42.6 | 65.98 | |
| BitDelta-InitialModel=Llama 2-70B Chat2024.02 | 41.63 | 66.01 | |
| MPT-7B-ChatModel=MPT-7B-Chat, Mode=Baseline2024.02 | 40.22 | 56.5 | |
| BitDeltaModel=MPT-7B-Chat2024.02 | 39.87 | 56.52 | |
| BitDelta-InitialModel=MPT-7B-Chat2024.02 | 38.96 | 57.11 | |
| MPT-7BModel=MPT-7B2024.02 | 33.37 | 37.95 |