List your company
ResearchFormerly WizWand

LLM Inference on Qwen3-8B 2k prompts Prefill-heavy workload

106,324Throughput (tok/s)v1
19,179.2841,803.3964,427.587,051.61May 30, 2026
Updated 5mo ago

Evaluation results

MethodLinks
v1Concurrency (c)=512, Hardware=4x H200, Data Parallelism (DP)=42026.05106,32480536.2
EB+Concurrency (c)=512, Hardware=4x H200, Data Parallelism (DP)=42026.05105,43284336.2
v1Concurrency (c)=256, Hardware=4x H200, Data Parallelism (DP)=42026.0588,48830423.4
EB+Concurrency (c)=256, Hardware=4x H200, Data Parallelism (DP)=42026.0587,26830723.8
2P+2DConcurrency (c)=128, Hardware=4x H2002026.0582,4006628.5
2P+2DConcurrency (c)=512, Hardware=4x H2002026.0578,4865,7418.2
2P+2DConcurrency (c)=256, Hardware=4x H2002026.0577,1172,6148.3
3P+1DConcurrency (c)=128, Hardware=4x H2002026.0570,25541213.1
v1Concurrency (c)=128, Hardware=4x H200, Data Parallelism (DP)=42026.0567,27813316.1
EB+Concurrency (c)=128, Hardware=4x H200, Data Parallelism (DP)=42026.0565,29712416.7
v1Concurrency (c)=512, Hardware=4x RTX PRO 6000, Data Parallelism (DP)=42026.0547,9471,14185.3
EB+Concurrency (c)=512, Hardware=4x RTX PRO 6000, Data Parallelism (DP)=42026.0547,3221,20586.1
2P+2DConcurrency (c)=256, Hardware=4x RTX PRO 60002026.0544,8923,36923.3
2P+2DConcurrency (c)=512, Hardware=4x RTX PRO 60002026.0543,3269,24523.9
2P+2DConcurrency (c)=128, Hardware=4x RTX PRO 60002026.0542,57747623.1
1P+3DConcurrency (c)=256, Hardware=4x H2002026.0540,6386,1036.4
1P+3DConcurrency (c)=128, Hardware=4x H2002026.0540,5352,7636.4
1P+3DConcurrency (c)=512, Hardware=4x H2002026.0539,92512,3576.4
v1Concurrency (c)=256, Hardware=4x RTX PRO 6000, Data Parallelism (DP)=42026.0537,62338057.8
EB+Concurrency (c)=256, Hardware=4x RTX PRO 6000, Data Parallelism (DP)=42026.0537,20838158.5
3P+1DConcurrency (c)=128, Hardware=4x RTX PRO 60002026.0531,49048232.6
EB+Concurrency (c)=128, Hardware=4x RTX PRO 6000, Data Parallelism (DP)=42026.0526,20717142.8
v1Concurrency (c)=128, Hardware=4x RTX PRO 6000, Data Parallelism (DP)=42026.0525,99418543
1P+3DConcurrency (c)=128, Hardware=4x RTX PRO 60002026.0523,2454,38514.6
1P+3DConcurrency (c)=256, Hardware=4x RTX PRO 60002026.0523,10210,28314.5
1P+3DConcurrency (c)=512, Hardware=4x RTX PRO 60002026.0522,53121,41214.6