Code Generation on MBPP (TPS, Speedup)
4,290TPSDomino
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| DominoModel=Qwen3-4B, Concurrency=32, Draft size=162026.05 | 4,290 | 1.9 | |
| DFlashModel=Qwen3-4B, Concurrency=32, Draft size=162026.05 | 4,088 | 1.8 | |
| DominoModel=Qwen3-4B, Concurrency=16, Draft size=162026.05 | 3,422 | 2.2 | |
| DFlashModel=Qwen3-4B, Concurrency=16, Draft size=162026.05 | 3,330 | 2.1 | |
| DominoModel=Qwen3-8B, Concurrency=32, Draft size=162026.05 | 3,027 | 2.1 | |
| DFlashModel=Qwen3-8B, Concurrency=32, Draft size=162026.05 | 2,800 | 2 | |
| DominoModel=Qwen3-8B, Concurrency=16, Draft size=162026.05 | 2,727 | 2.5 | |
| DominoModel=Qwen3-4B, Concurrency=8, Draft size=162026.05 | 2,651 | 2.6 | |
| DFlashModel=Qwen3-4B, Concurrency=8, Draft size=162026.05 | 2,501 | 2.5 | |
| DFlashModel=Qwen3-8B, Concurrency=16, Draft size=162026.05 | 2,487 | 2.3 | |
| BaselineModel=Qwen3-4B, Concurrency=322026.05 | 2,300 | — | |
| DominoModel=Qwen3-8B, Concurrency=8, Draft size=162026.05 | 2,035 | 3.2 | |
| EAGLE-3Model=Qwen3-4B, Concurrency=32, Draft size=162026.05 | 1,892 | 0.8 | |
| DFlashModel=Qwen3-8B, Concurrency=8, Draft size=162026.05 | 1,889 | 3 | |
| DominoModel=Qwen3-4B, Concurrency=4, Draft size=162026.05 | 1,654 | 3 | |
| DFlashModel=Qwen3-4B, Concurrency=4, Draft size=162026.05 | 1,650 | 3 | |
| EAGLE-3Model=Qwen3-4B, Concurrency=16, Draft size=162026.05 | 1,597 | 1 | |
| BaselineModel=Qwen3-4B, Concurrency=162026.05 | 1,586 | — | |
| BaselineModel=Qwen3-8B, Concurrency=322026.05 | 1,428 | — | |
| EAGLE-3Model=Qwen3-8B, Concurrency=32, Draft size=162026.05 | 1,291 | 0.9 | |
| DominoModel=Qwen3-8B, Concurrency=4, Draft size=162026.05 | 1,256 | 3.6 | |
| EAGLE-3Model=Qwen3-4B, Concurrency=8, Draft size=162026.05 | 1,186 | 1.2 | |
| EAGLE-3Model=Qwen3-8B, Concurrency=16, Draft size=162026.05 | 1,180 | 1.1 | |
| DFlashModel=Qwen3-8B, Concurrency=4, Draft size=162026.05 | 1,169 | 3.3 | |
| BaselineModel=Qwen3-8B, Concurrency=162026.05 | 1,076 | — | |
| BaselineModel=Qwen3-4B, Concurrency=82026.05 | 1,002 | — | |
| EAGLE-3Model=Qwen3-4B, Concurrency=32, Draft size=602026.05 | 987 | 0.4 | |
| DominoModel=Qwen3-4B, Concurrency=2, Draft size=162026.05 | 968 | 3.3 | |
| DFlashModel=Qwen3-4B, Concurrency=2, Draft size=162026.05 | 914 | 3.1 | |
| EAGLE-3Model=Qwen3-4B, Concurrency=16, Draft size=602026.05 | 899 | 0.6 | |
| EAGLE-3Model=Qwen3-8B, Concurrency=8, Draft size=162026.05 | 886 | 1.4 | |
| EAGLE-3Model=Qwen3-4B, Concurrency=8, Draft size=602026.05 | 793 | 0.8 | |
| EAGLE-3Model=Qwen3-4B, Concurrency=4, Draft size=162026.05 | 740 | 1.4 | |
| DominoModel=Qwen3-8B, Concurrency=2, Draft size=162026.05 | 701 | 3.8 | |
| DFlashModel=Qwen3-8B, Concurrency=2, Draft size=162026.05 | 649 | 3.6 | |
| BaselineModel=Qwen3-8B, Concurrency=82026.05 | 635 | — | |
| EAGLE-3Model=Qwen3-8B, Concurrency=32, Draft size=602026.05 | 631 | 0.4 | |
| EAGLE-3Model=Qwen3-4B, Concurrency=4, Draft size=602026.05 | 610 | 1.1 | |
| EAGLE-3Model=Qwen3-8B, Concurrency=16, Draft size=602026.05 | 574 | 0.5 | |
| BaselineModel=Qwen3-4B, Concurrency=42026.05 | 544 | — | |
| EAGLE-3Model=Qwen3-8B, Concurrency=4, Draft size=162026.05 | 525 | 1.5 | |
| EAGLE-3Model=Qwen3-8B, Concurrency=8, Draft size=602026.05 | 525 | 0.8 | |
| EAGLE-3Model=Qwen3-8B, Concurrency=4, Draft size=602026.05 | 438 | 1.2 | |
| EAGLE-3Model=Qwen3-4B, Concurrency=2, Draft size=602026.05 | 414 | 1.4 | |
| EAGLE-3Model=Qwen3-4B, Concurrency=2, Draft size=162026.05 | 407 | 1.4 | |
| BaselineModel=Qwen3-8B, Concurrency=42026.05 | 352 | — | |
| EAGLE-3Model=Qwen3-8B, Concurrency=2, Draft size=602026.05 | 293 | 1.6 | |
| BaselineModel=Qwen3-4B, Concurrency=22026.05 | 291 | — | |
| EAGLE-3Model=Qwen3-8B, Concurrency=2, Draft size=162026.05 | 290 | 1.6 | |
| BaselineModel=Qwen3-8B, Concurrency=22026.05 | 183 | — |