Mathematics on MATH 500 (Throughput and Latency)
10,943Throughput (tok/s)Draft-OPD
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Draft-OPDBackbone Model=Qwen3-4B (Enable Thinking), Concurrency=32, Inference Engine=SGLang2026.05 | 10,943 | 9 | 6.68 | |
| DFlashBackbone Model=Qwen3-4B (Enable Thinking), Concurrency=32, Inference Engine=SGLang2026.05 | 10,062 | — | 6.17 | |
| Draft-OPDBackbone Model=Qwen3-4B (Enable Thinking), Concurrency=16, Inference Engine=SGLang2026.05 | 8,593 | 7 | 6.68 | |
| DFlashBackbone Model=Qwen3-4B (Enable Thinking), Concurrency=16, Inference Engine=SGLang2026.05 | 8,004 | — | 6.17 | |
| Draft-OPDBackbone Model=Qwen3-8B (Enable Thinking), Concurrency=32, Inference Engine=SGLang2026.05 | 7,940 | 13 | 6.64 | |
| DFlashBackbone Model=Qwen3-8B (Enable Thinking), Concurrency=32, Inference Engine=SGLang2026.05 | 6,991 | — | 5.99 | |
| Draft-OPDBackbone Model=Qwen3-8B (Enable Thinking), Concurrency=16, Inference Engine=SGLang2026.05 | 6,666 | 12 | 6.64 | |
| DFlashBackbone Model=Qwen3-8B (Enable Thinking), Concurrency=16, Inference Engine=SGLang2026.05 | 5,958 | — | 5.99 | |
| Draft-OPDBackbone Model=Qwen3-4B (Enable Thinking), Concurrency=8, Inference Engine=SGLang2026.05 | 5,603 | 7 | 6.68 | |
| DFlashBackbone Model=Qwen3-4B (Enable Thinking), Concurrency=8, Inference Engine=SGLang2026.05 | 5,234 | — | 6.17 | |
| Draft-OPDBackbone Model=Qwen3-30B-A3B-Thinking-2507, Concurrency=32, Inference Engine=SGLang2026.05 | 5,007 | 12 | 5.95 | |
| Draft-OPDBackbone Model=Qwen3-8B (Enable Thinking), Concurrency=8, Inference Engine=SGLang2026.05 | 4,691 | 13 | 6.64 | |
| DFlashBackbone Model=Qwen3-30B-A3B-Thinking-2507, Concurrency=32, Inference Engine=SGLang2026.05 | 4,462 | — | 5.44 | |
| DFlashBackbone Model=Qwen3-8B (Enable Thinking), Concurrency=8, Inference Engine=SGLang2026.05 | 4,154 | — | 5.99 | |
| Draft-OPDBackbone Model=Qwen3-4B (Enable Thinking), Concurrency=4, Inference Engine=SGLang2026.05 | 3,413 | 8 | 6.68 | |
| Draft-OPDBackbone Model=Qwen3-30B-A3B-Thinking-2507, Concurrency=16, Inference Engine=SGLang2026.05 | 3,405 | 12 | 5.95 | |
| DFlashBackbone Model=Qwen3-4B (Enable Thinking), Concurrency=4, Inference Engine=SGLang2026.05 | 3,157 | — | 6.17 | |
| DFlashBackbone Model=Qwen3-30B-A3B-Thinking-2507, Concurrency=16, Inference Engine=SGLang2026.05 | 3,020 | — | 5.44 | |
| Draft-OPDBackbone Model=Qwen3-8B (Enable Thinking), Concurrency=4, Inference Engine=SGLang2026.05 | 2,721 | 14 | 6.64 | |
| DFlashBackbone Model=Qwen3-8B (Enable Thinking), Concurrency=4, Inference Engine=SGLang2026.05 | 2,374 | — | 5.99 | |
| Draft-OPDBackbone Model=Qwen3-30B-A3B-Thinking-2507, Concurrency=8, Inference Engine=SGLang2026.05 | 2,243 | 11 | 5.95 | |
| DFlashBackbone Model=Qwen3-30B-A3B-Thinking-2507, Concurrency=8, Inference Engine=SGLang2026.05 | 2,009 | — | 5.44 | |
| Draft-OPDBackbone Model=Qwen3-30B-A3B-Thinking-2507, Concurrency=4, Inference Engine=SGLang2026.05 | 1,303 | 11 | 5.95 | |
| DFlashBackbone Model=Qwen3-30B-A3B-Thinking-2507, Concurrency=4, Inference Engine=SGLang2026.05 | 1,176 | — | 5.44 | |
| Draft-OPDBackbone Model=Qwen3-4B (Enable Thinking), Concurrency=1, Inference Engine=SGLang2026.05 | 1,036 | 9 | 6.68 | |
| DFlashBackbone Model=Qwen3-4B (Enable Thinking), Concurrency=1, Inference Engine=SGLang2026.05 | 949 | — | 6.17 | |
| Draft-OPDBackbone Model=Qwen3-8B (Enable Thinking), Concurrency=1, Inference Engine=SGLang2026.05 | 787 | 12 | 6.64 | |
| DFlashBackbone Model=Qwen3-8B (Enable Thinking), Concurrency=1, Inference Engine=SGLang2026.05 | 703 | — | 5.99 | |
| Draft-OPDBackbone Model=Qwen3-30B-A3B-Thinking-2507, Concurrency=1, Inference Engine=SGLang2026.05 | 477 | 14 | 5.95 | |
| DFlashBackbone Model=Qwen3-30B-A3B-Thinking-2507, Concurrency=1, Inference Engine=SGLang2026.05 | 417 | — | 5.44 |