Large Language Model Throughput on ShareGPT v3 benchmark leaderboard · SOTA2 Research

Large Language Model Throughput on ShareGPT v3

8.85Throughput (req/s)vLLM-RA

Evaluation Results

Method	Links
vLLM-RA2024.02		8.85
vLLM-RA2024.02		8.76
vLLM-PS2024.02		6.29
vLLM-RA2024.02		5.9
vLLM-RA2024.02		5.85
vLLM-PS2024.02		5.4
vLLM2024.02		5.03
vLLM-PS2024.02		4.92
vLLM-PS2024.02		4.82
vLLM2024.02		3.68
vLLM-RA2024.02		3.65
vLLM-RA2024.02		3.64
vLLM2024.02		3.54
vLLM-PS2024.02		3.01
vLLM2024.02		2.6
vLLM-PS2024.02		2.55
vLLM2024.02		2.15
vLLM-RA2024.02		1.71
vLLM-RA2024.02		1.69
vLLM2024.02		1.52
vLLM-PS2024.02		1.44
vLLM-PS2024.02		1.23
vLLM2024.02		0.99
vLLM2024.02		0.66