Mathematical Reasoning on GSM8K (Adjusted Average)
56.79GSM8K AccuracyBitDelta-Initial
Evaluation results
| Method | Links | ||
|---|---|---|---|
| BitDelta-InitialModel=Solar-0-70B2024.02 | 56.79 | 73.14 | |
| BitDeltaModel=Solar-0-70B2024.02 | 56.63 | 73.57 | |
| Solar-0-70BModel=Solar-0-70B, Mode=Baseline2024.02 | 56.18 | 73.77 | |
| Dolphin 2.2.1Model=Dolphin 2.2.1, Mode=Baseline2024.02 | 54.28 | 67.31 | |
| BitDeltaModel=Dolphin 2.2.12024.02 | 52.84 | 66.97 | |
| Llama 2-70BModel=Llama 2-70B2024.02 | 52.69 | 71.81 | |
| BitDelta-InitialModel=Dolphin 2.2.12024.02 | 50.27 | 67.58 | |
| BitDeltaModel=Llama 2-70B Chat2024.02 | 48.82 | 69.14 | |
| Llama 2-70B ChatModel=Llama 2-70B Chat, Mode=Baseline2024.02 | 47.61 | 68.82 | |
| BitDelta-InitialModel=Llama 2-70B Chat2024.02 | 42.38 | 66.01 | |
| BitDelta-InitialModel=Zephyr-7B-β2024.02 | 40.26 | 66.12 | |
| BitDelta-InitialModel=Mistral-7B v0.1 Instruct2024.02 | 38.82 | 63.83 | |
| Mistral-7B v0.1Model=Mistral-7B v0.12024.02 | 37.76 | 65.98 | |
| Zephyr-7B-βModel=Zephyr-7B-β, Mode=Baseline2024.02 | 34.34 | 65.22 | |
| Mistral-7B v0.1 InstructModel=Mistral-7B v0.1 Instruct, Mode=Baseline2024.02 | 32.75 | 60.36 | |
| BitDeltaModel=Zephyr-7B-β2024.02 | 31.92 | 66.2 | |
| BitDeltaModel=Mistral-7B v0.1 Instruct2024.02 | 31.54 | 61.1 | |
| BitDelta-InitialModel=MPT-7B-Chat2024.02 | 10.01 | 57.11 | |
| BitDeltaModel=MPT-7B-Chat2024.02 | 8.11 | 56.52 | |
| MPT-7B-ChatModel=MPT-7B-Chat, Mode=Baseline2024.02 | 7.96 | 56.5 | |
| MPT-7BModel=MPT-7B2024.02 | 6.22 | 37.95 |