Language Understanding on MMLU (Accuracy, Throughput, and Latency)
84.9AccuracyVerify-Only
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Verify-OnlyDraft-Target Model Pair=Qwen 2.5 72B – Qwen 2.5 7B2025.12 | 84.9 | 10.952 | 0.408 | 1.042 | |
| TargetDraft-Target Model Pair=Qwen 2.5 72B – Qwen 2.5 7B2025.12 | 84.5 | 4.69 | — | 0.446 | |
| Gen-OnlyDraft-Target Model Pair=Qwen 2.5 72B – Qwen 2.5 7B2025.12 | 84.1 | 10.735 | — | 1.021 | |
| VanillaDraft-Target Model Pair=Llama 3.1 70B – Llama 3.2 1B2025.12 | 83.8 | 8.746 | — | 1 | |
| VanillaDraft-Target Model Pair=Qwen 2.5 72B – Qwen 2.5 7B2025.12 | 83.7 | 10.511 | — | 1 | |
| AssistedGenDraft-Target Model Pair=Qwen 2.5 72B – Qwen 2.5 7B2025.12 | 83.7 | 11.436 | — | 1.088 | |
| TargetDraft-Target Model Pair=Llama 3.1 70B – Llama 3.1 8B2025.12 | 83.6 | 4.376 | — | 0.553 | |
| TargetDraft-Target Model Pair=Llama 3.1 70B – Llama 3.2 1B2025.12 | 83.6 | 4.376 | — | 0.5 | |
| VanillaDraft-Target Model Pair=Llama 3.1 70B – Llama 3.1 8B2025.12 | 83.5 | 7.912 | — | 1 | |
| AssistedGenDraft-Target Model Pair=Llama 3.1 70B – Llama 3.1 8B2025.12 | 83.5 | 7.97 | — | 1.007 | |
| Gen-OnlyDraft-Target Model Pair=Llama 3.1 70B – Llama 3.2 1B2025.12 | 83.5 | 9.085 | — | 1.039 | |
| AdaSDDraft-Target Model Pair=Qwen 2.5 72B – Qwen 2.5 7B2025.12 | 83.5 | 11.395 | 0.407 | 1.084 | |
| Verify-OnlyDraft-Target Model Pair=Llama 3.1 70B – Llama 3.1 8B2025.12 | 83.3 | 8.481 | 0.418 | 1.072 | |
| AssistedGenDraft-Target Model Pair=Llama 3.1 70B – Llama 3.2 1B2025.12 | 83.3 | 8.92 | — | 1.02 | |
| Gen-OnlyDraft-Target Model Pair=Llama 3.1 70B – Llama 3.1 8B2025.12 | 83.1 | 8.023 | — | 1.014 | |
| AdaSDDraft-Target Model Pair=Llama 3.1 70B – Llama 3.1 8B2025.12 | 82.2 | 8.623 | 0.419 | 1.09 | |
| Verify-OnlyDraft-Target Model Pair=Llama 3.1 70B – Llama 3.2 1B2025.12 | 81.6 | 9.385 | 0.465 | 1.073 | |
| AdaSDDraft-Target Model Pair=Llama 3.1 70B – Llama 3.2 1B2025.12 | 81.3 | 9.937 | 0.463 | 1.136 | |
| Cosine (7.5, 0)Schedule Type=Cosine, Thresholds (τ_high, τ_low)=(7.5, 0)2025.12 | 80.81 | — | — | 1.56 | |
| Linear (7.5, 0)Schedule Type=Linear, Thresholds (τ_high, τ_low)=(7.5, 0)2025.12 | 80.81 | — | — | 1.56 | |
| Exp-k=2 (7.5, 0)Schedule Type=Exponential, k=2, Thresholds (τ_high, τ_low)=(7.5, 0)2025.12 | 80.77 | — | — | 1.62 | |
| Linear (7.5, 2.5)Schedule Type=Linear, Thresholds (τ_high, τ_low)=(7.5, 2.5)2025.12 | 80.76 | — | — | 1.43 | |
| Cosine (7.5, 2.5)Schedule Type=Cosine, Thresholds (τ_high, τ_low)=(7.5, 2.5)2025.12 | 80.75 | — | — | 1.43 | |
| BaselineSchedule=Standard2025.12 | 80.69 | — | — | 1 | |
| Exp-k=2 (7.5, 2.5)Schedule Type=Exponential, k=2, Thresholds (τ_high, τ_low)=(7.5, 2.5)2025.12 | 80.69 | — | — | 1.54 | |
| Exp-k=16 (7.5, 0)Schedule Type=Exponential, k=16, Thresholds (τ_high, τ_low)=(7.5, 0)2025.12 | 79.31 | — | — | 5 | |
| Exp-k=16 (7.5, 2.5)Schedule Type=Exponential, k=16, Thresholds (τ_high, τ_low)=(7.5, 2.5)2025.12 | 79.31 | — | — | 2.5 | |
| ProphetSchedule=Prophet2025.12 | 78.73 | — | — | 1.14 | |
| DraftDraft-Target Model Pair=Qwen 2.5 72B – Qwen 2.5 7B2025.12 | 72.5 | 38.544 | — | 3.667 | |
| DraftDraft-Target Model Pair=Llama 3.1 70B – Llama 3.1 8B2025.12 | 69.5 | 33.274 | — | 4.206 | |
| DraftDraft-Target Model Pair=Llama 3.1 70B – Llama 3.2 1B2025.12 | 45.3 | 95.141 | — | 10.878 |