List your company
ResearchFormerly WizWand

Large Language Model Inference on GPQA

2,341.3ThroughputSpecBundle
104.468685.1841,265.91,846.616Mar 19, 2026
Updated 7mo ago

Evaluation results

MethodLinks
SpecBundleTarget Model=Qwen-30B-A3B, Draft Model=SpecBundle, #GPUs=42026.032,341.31.66
SpecBundleTarget Model=Llama-4-Scout, Draft Model=SpecBundle, #GPUs=82026.031,502.22.78
Standard InferenceTarget Model=Qwen-30B-A3B, Draft Model=-, #GPUs=42026.031,410.41
SpecBundleTarget Model=Llama-3.3-70B, Draft Model=SpecBundle, #GPUs=42026.031,405.12.44
EAGLE3Target Model=Llama-4-Scout, Draft Model=Existing, #GPUs=82026.031,405.12.6
SpecBundleTarget Model=Ling-Flash-V2, Draft Model=SpecBundle, #GPUs=82026.031,185.71.49
EAGLE3Target Model=Llama-3.3-70B, Draft Model=Existing, #GPUs=42026.031,0491.82
SpecBundleTarget Model=Qwen-235B-A22B, Draft Model=SpecBundle, #GPUs=82026.03826.51.47
SpecBundleTarget Model=Kimi-K2, Draft Model=SpecBundle, #GPUs=82026.03811.41.61
Standard InferenceTarget Model=Ling-Flash-V2, Draft Model=-, #GPUs=82026.03794.11
EAGLE3Target Model=Qwen-235B-A22B, Draft Model=Existing, #GPUs=82026.03716.71.27
Standard InferenceTarget Model=Llama-3.3-70B, Draft Model=-, #GPUs=42026.03575.71
Standard InferenceTarget Model=Qwen-235B-A22B, Draft Model=-, #GPUs=82026.03563.21
Standard InferenceTarget Model=Llama-4-Scout, Draft Model=-, #GPUs=82026.035411
SpecBundleTarget Model=Llama-3.1-8B, Draft Model=SpecBundle, #GPUs=12026.03514.22.7
Standard InferenceTarget Model=Kimi-K2, Draft Model=-, #GPUs=82026.03505.41
EAGLE3Target Model=Llama-3.1-8B, Draft Model=Existing, #GPUs=12026.03438.12.3
Standard InferenceTarget Model=Llama-3.1-8B, Draft Model=-, #GPUs=12026.03190.51