Question Answering on TruthfulQA (TruthfulQA/Adjusted Average Metrics)
50.38TruthfulQA ScoreBaseline
Evaluation results
| Method | Links | ||
|---|---|---|---|
| BaselineBackbone Model=Vicuna-7B v1.5 16k2024.02 | 50.38 | 57.5 | |
| BaselineBackbone Model=Vicuna-13B v1.5 16k2024.02 | 50.38 | 57.5 | |
| BitDeltaBackbone Model=Vicuna-7B v1.5 16k2024.02 | 48.75 | 57.64 | |
| BitDeltaBackbone Model=Vicuna-13B v1.5 16k2024.02 | 48.75 | 57.64 | |
| BaselineBackbone Model=WizardLM-13B v1.22024.02 | 47.17 | 61.61 | |
| BitDeltaBackbone Model=WizardLM-13B v1.22024.02 | 46.67 | 61.86 | |
| BitDelta-InitialBackbone Model=Vicuna-7B v1.5 16k2024.02 | 45.58 | 58.51 | |
| BaselineBackbone Model=Llama 2-7B Chat2024.02 | 45.32 | 59.81 | |
| BitDeltaBackbone Model=Llama 2-7B Chat2024.02 | 44.95 | 59.88 | |
| BitDelta-InitialBackbone Model=WizardLM-13B v1.22024.02 | 44.89 | 61.91 | |
| BaselineBackbone Model=Llama 2-13B Chat2024.02 | 43.95 | 63.99 | |
| BitDeltaBackbone Model=Llama 2-13B Chat2024.02 | 43.47 | 63.96 | |
| BitDelta-InitialBackbone Model=Llama 2-13B Chat2024.02 | 41.7 | 64.25 | |
| BitDelta-InitialBackbone Model=Vicuna-13B v1.5 16k2024.02 | 41.7 | 64.25 | |
| BitDelta-InitialBackbone Model=Llama 2-7B Chat2024.02 | 41.1 | 60.7 | |
| Llama 2-7BModel Type=Base Model2024.02 | 38.96 | 60.53 | |
| Llama 2-13BModel Type=Base Model2024.02 | 36.9 | 64.68 | |
| LESASFT=true, training_steps=6k2025.02 | 22.28 | 31.57 | |
| LLaMA ProSFT=true, training_steps=6k2025.02 | 21.91 | 24.38 | |
| SOLARSFT=true, training_steps=6k2025.02 | 19.34 | 26.47 |