List your company
ResearchFormerly WizWand

LLM Inference on Qwen3-8B (2k prompts Balanced workload)

61,106Throughput (tok/s)v1
6,939.6821,002.0935,064.549,126.91May 30, 2026
Updated 5mo ago

Evaluation results

MethodLinks
v1Concurrency (c)=512, Hardware=4x H200, Data Parallelism (DP)=42026.0561,10646414.7
EB+Concurrency (c)=512, Hardware=4x H200, Data Parallelism (DP)=42026.0560,98841314.9
2P+2DConcurrency (c)=512, Hardware=4x H2002026.0548,8951,12817.3
1P+3DConcurrency (c)=256, Hardware=4x H2002026.0547,4336818.9
1P+3DConcurrency (c)=512, Hardware=4x H2002026.0546,2615,7539.2
EB+Concurrency (c)=256, Hardware=4x H200, Data Parallelism (DP)=42026.0543,72915210.9
v1Concurrency (c)=256, Hardware=4x H200, Data Parallelism (DP)=42026.0543,64815810.9
2P+2DConcurrency (c)=256, Hardware=4x H2002026.0542,01639510.9
1P+3DConcurrency (c)=128, Hardware=4x H2002026.0531,3412497.5
2P+2DConcurrency (c)=128, Hardware=4x H2002026.0529,3881788.2
EB+Concurrency (c)=128, Hardware=4x H200, Data Parallelism (DP)=42026.0528,686748.6
v1Concurrency (c)=128, Hardware=4x H200, Data Parallelism (DP)=42026.0528,646758.6
3P+1DConcurrency (c)=256, Hardware=4x H2002026.0526,45559617.6
1P+3DConcurrency (c)=512, Hardware=4x RTX PRO 60002026.0525,0804,01329.9
EB+Concurrency (c)=512, Hardware=4x RTX PRO 6000, Data Parallelism (DP)=42026.0524,03769738.6
v1Concurrency (c)=512, Hardware=4x RTX PRO 6000, Data Parallelism (DP)=42026.0523,91469738.7
3P+1DConcurrency (c)=128, Hardware=4x H2002026.0521,96323311
2P+2DConcurrency (c)=512, Hardware=4x RTX PRO 60002026.0520,3971,67643.4
1P+3DConcurrency (c)=256, Hardware=4x RTX PRO 60002026.0519,83184422.9
2P+2DConcurrency (c)=256, Hardware=4x RTX PRO 60002026.0517,55150526.9
v1Concurrency (c)=256, Hardware=4x RTX PRO 6000, Data Parallelism (DP)=42026.0516,91323528.6
EB+Concurrency (c)=256, Hardware=4x RTX PRO 6000, Data Parallelism (DP)=42026.0516,87022928.7
1P+3DConcurrency (c)=128, Hardware=4x RTX PRO 60002026.0512,62931619.2
2P+2DConcurrency (c)=128, Hardware=4x RTX PRO 60002026.0511,64022921.1
3P+1DConcurrency (c)=256, Hardware=4x RTX PRO 60002026.0511,40266442.1
EB+Concurrency (c)=128, Hardware=4x RTX PRO 6000, Data Parallelism (DP)=42026.0511,06311622.5
v1Concurrency (c)=128, Hardware=4x RTX PRO 6000, Data Parallelism (DP)=42026.0511,01312122.6
3P+1DConcurrency (c)=128, Hardware=4x RTX PRO 60002026.059,02327227.3