List your company
ResearchFormerly WizWand

LLM Inference on Merge

1.16Goodput (req/s)AugServe
-0.00480.29760.60.9024Dec 3, 2025
Updated 8mo ago

Evaluation results

MethodLinks
AugServeRequest rate=3, CV=1, Model=OPT-13B, GPU=H8002025.121.16
AugServeRequest rate=2, CV=1.5, Model=OPT-13B, GPU=H8002025.121.15
AugServeRequest rate=2, CV=1, Model=OPT-13B, GPU=H8002025.120.87
AugServeRequest rate=3, CV=1.5, Model=OPT-13B, GPU=H8002025.120.58
AugServeRequest rate=2, CV=2, Model=OPT-13B, GPU=H8002025.120.47
AugServeRequest rate=3, CV=2, Model=OPT-13B, GPU=H8002025.120.35
InferCeptRequest rate=2, CV=1, Model=OPT-13B, GPU=H8002025.120.28
vLLMRequest rate=2, CV=1, Model=OPT-13B, GPU=H8002025.120.22
InferCeptRequest rate=3, CV=1, Model=OPT-13B, GPU=H8002025.120.2
vLLMRequest rate=3, CV=1, Model=OPT-13B, GPU=H8002025.120.15
InferCeptRequest rate=2, CV=1.5, Model=OPT-13B, GPU=H8002025.120.13
InferCeptRequest rate=3, CV=1.5, Model=OPT-13B, GPU=H8002025.120.11
vLLMRequest rate=2, CV=1.5, Model=OPT-13B, GPU=H8002025.120.1
InferCeptRequest rate=2, CV=2, Model=OPT-13B, GPU=H8002025.120.08
vLLMRequest rate=3, CV=1.5, Model=OPT-13B, GPU=H8002025.120.07
InferCeptRequest rate=3, CV=2, Model=OPT-13B, GPU=H8002025.120.06
vLLMRequest rate=2, CV=2, Model=OPT-13B, GPU=H8002025.120.05
vLLMRequest rate=3, CV=2, Model=OPT-13B, GPU=H8002025.120.04