List your company
ResearchFormerly WizWand

Code Generation on MBPP (TPS, Speedup)

4,290TPSDomino
18.721,127.612,236.53,345.39May 28, 2026
Updated 5mo ago

Evaluation results

MethodLinks
DominoModel=Qwen3-4B, Concurrency=32, Draft size=162026.054,2901.9
DFlashModel=Qwen3-4B, Concurrency=32, Draft size=162026.054,0881.8
DominoModel=Qwen3-4B, Concurrency=16, Draft size=162026.053,4222.2
DFlashModel=Qwen3-4B, Concurrency=16, Draft size=162026.053,3302.1
DominoModel=Qwen3-8B, Concurrency=32, Draft size=162026.053,0272.1
DFlashModel=Qwen3-8B, Concurrency=32, Draft size=162026.052,8002
DominoModel=Qwen3-8B, Concurrency=16, Draft size=162026.052,7272.5
DominoModel=Qwen3-4B, Concurrency=8, Draft size=162026.052,6512.6
DFlashModel=Qwen3-4B, Concurrency=8, Draft size=162026.052,5012.5
DFlashModel=Qwen3-8B, Concurrency=16, Draft size=162026.052,4872.3
BaselineModel=Qwen3-4B, Concurrency=322026.052,300—
DominoModel=Qwen3-8B, Concurrency=8, Draft size=162026.052,0353.2
EAGLE-3Model=Qwen3-4B, Concurrency=32, Draft size=162026.051,8920.8
DFlashModel=Qwen3-8B, Concurrency=8, Draft size=162026.051,8893
DominoModel=Qwen3-4B, Concurrency=4, Draft size=162026.051,6543
DFlashModel=Qwen3-4B, Concurrency=4, Draft size=162026.051,6503
EAGLE-3Model=Qwen3-4B, Concurrency=16, Draft size=162026.051,5971
BaselineModel=Qwen3-4B, Concurrency=162026.051,586—
BaselineModel=Qwen3-8B, Concurrency=322026.051,428—
EAGLE-3Model=Qwen3-8B, Concurrency=32, Draft size=162026.051,2910.9
DominoModel=Qwen3-8B, Concurrency=4, Draft size=162026.051,2563.6
EAGLE-3Model=Qwen3-4B, Concurrency=8, Draft size=162026.051,1861.2
EAGLE-3Model=Qwen3-8B, Concurrency=16, Draft size=162026.051,1801.1
DFlashModel=Qwen3-8B, Concurrency=4, Draft size=162026.051,1693.3
BaselineModel=Qwen3-8B, Concurrency=162026.051,076—
BaselineModel=Qwen3-4B, Concurrency=82026.051,002—
EAGLE-3Model=Qwen3-4B, Concurrency=32, Draft size=602026.059870.4
DominoModel=Qwen3-4B, Concurrency=2, Draft size=162026.059683.3
DFlashModel=Qwen3-4B, Concurrency=2, Draft size=162026.059143.1
EAGLE-3Model=Qwen3-4B, Concurrency=16, Draft size=602026.058990.6
EAGLE-3Model=Qwen3-8B, Concurrency=8, Draft size=162026.058861.4
EAGLE-3Model=Qwen3-4B, Concurrency=8, Draft size=602026.057930.8
EAGLE-3Model=Qwen3-4B, Concurrency=4, Draft size=162026.057401.4
DominoModel=Qwen3-8B, Concurrency=2, Draft size=162026.057013.8
DFlashModel=Qwen3-8B, Concurrency=2, Draft size=162026.056493.6
BaselineModel=Qwen3-8B, Concurrency=82026.05635—
EAGLE-3Model=Qwen3-8B, Concurrency=32, Draft size=602026.056310.4
EAGLE-3Model=Qwen3-4B, Concurrency=4, Draft size=602026.056101.1
EAGLE-3Model=Qwen3-8B, Concurrency=16, Draft size=602026.055740.5
BaselineModel=Qwen3-4B, Concurrency=42026.05544—
EAGLE-3Model=Qwen3-8B, Concurrency=4, Draft size=162026.055251.5
EAGLE-3Model=Qwen3-8B, Concurrency=8, Draft size=602026.055250.8
EAGLE-3Model=Qwen3-8B, Concurrency=4, Draft size=602026.054381.2
EAGLE-3Model=Qwen3-4B, Concurrency=2, Draft size=602026.054141.4
EAGLE-3Model=Qwen3-4B, Concurrency=2, Draft size=162026.054071.4
BaselineModel=Qwen3-8B, Concurrency=42026.05352—
EAGLE-3Model=Qwen3-8B, Concurrency=2, Draft size=602026.052931.6
BaselineModel=Qwen3-4B, Concurrency=22026.05291—
EAGLE-3Model=Qwen3-8B, Concurrency=2, Draft size=162026.052901.6
BaselineModel=Qwen3-8B, Concurrency=22026.05183—