Mathematical Reasoning on GSM8K (GSM8K Score & Adjusted Average)
42.38GSM8K ScoreBaseline
Evaluation results
| Method | Links | ||
|---|---|---|---|
| BaselineBackbone Model=WizardLM-13B v1.22024.02 | 42.38 | 61.61 | |
| BitDelta-InitialBackbone Model=WizardLM-13B v1.22024.02 | 42.08 | 61.91 | |
| BitDeltaBackbone Model=WizardLM-13B v1.22024.02 | 41.62 | 61.86 | |
| BitDelta-InitialBackbone Model=Llama 2-13B Chat2024.02 | 33.36 | 64.25 | |
| BaselineBackbone Model=Llama 2-13B Chat2024.02 | 33.13 | 63.99 | |
| BitDeltaBackbone Model=Llama 2-13B Chat2024.02 | 31.92 | 63.96 | |
| BaselineBackbone Model=Vicuna-13B v1.5 16k2024.02 | 29.72 | 57.5 | |
| BitDeltaBackbone Model=Vicuna-13B v1.5 16k2024.02 | 28.73 | 57.64 | |
| BitDelta-InitialBackbone Model=Vicuna-13B v1.5 16k2024.02 | 26.76 | 64.25 | |
| BaselineBackbone Model=Llama 2-7B Chat2024.02 | 22.74 | 59.81 | |
| Llama 2-13BModel Type=Base Model2024.02 | 22.74 | 64.68 | |
| BitDeltaBackbone Model=Llama 2-7B Chat2024.02 | 20.24 | 59.88 | |
| BitDelta-InitialBackbone Model=Llama 2-7B Chat2024.02 | 18.27 | 60.7 | |
| BitDeltaBackbone Model=Vicuna-7B v1.5 16k2024.02 | 14.48 | 57.64 | |
| BaselineBackbone Model=Vicuna-7B v1.5 16k2024.02 | 14.18 | 57.5 | |
| BitDelta-InitialBackbone Model=Vicuna-7B v1.5 16k2024.02 | 13.95 | 58.51 | |
| Llama 2-7BModel Type=Base Model2024.02 | 13.57 | 60.53 |