LLM Workflow Orchestration on Representative Workflows MMLU TAT-QA Amazon
1Avg Relative Latency (vs Helium)Helium
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| HeliumLLM Model=Qwen3-8B, Hardware=2x NVIDIA H100 NVL GPU, Sampling=Greedy2026.03 | 1 | 1 | 1 | |
| OpWiseLLM Model=Qwen3-8B, Hardware=2x NVIDIA H100 NVL GPU, Sampling=Greedy2026.03 | 1.25 | 1.02 | 1.58 | |
| LangGraphLLM Model=Qwen3-8B, Hardware=2x NVIDIA H100 NVL GPU, Sampling=Greedy2026.03 | 1.28 | 1.09 | 1.83 | |
| KVFlowLLM Model=Qwen3-8B, Hardware=2x NVIDIA H100 NVL GPU, Sampling=Greedy2026.03 | 1.32 | 1.14 | 1.56 | |
| ParrotLLM Model=Qwen3-8B, Hardware=2x NVIDIA H100 NVL GPU, Sampling=Greedy2026.03 | 1.44 | 1.02 | 2.21 | |
| AgentScopeLLM Model=Qwen3-8B, Hardware=2x NVIDIA H100 NVL GPU, Sampling=Greedy2026.03 | 2.1 | 1.07 | 4.32 | |
| vLLMLLM Model=Qwen3-8B, Hardware=2x NVIDIA H100 NVL GPU, Sampling=Greedy2026.03 | 66.27 | 38.18 | 100.92 |