# LLM Workflow Orchestration on Representative Workflows MMLU TAT-QA Amazon

Source: [LLM Workflow Orchestration on Representative Workflows MMLU TAT-QA Amazon](<https://www.sota2.com/research/sota/llm-workflow-orchestration-on-representative-workflows-mmlu-tat-qa-amazon>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-18T08:07:37.863Z

Evaluation of LLM workflow orchestration performance using the Representative Workflows MMLU TAT-QA Amazon dataset, focusing on relative latency metrics compared to a baseline (Helium).

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | LLM Workflow Orchestration |
| Dataset (including any stated split) | Representative Workflows MMLU TAT-QA Amazon |
| Metric | Avg Relative Latency (vs Helium) |
| Unit | SCALAR |
| Direction | Lower is better |
| Results loaded | 7 of 7 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Avg Relative Latency (vs Helium)](<https://www.sota2.com/research/sota/llm-workflow-orchestration-on-representative-workflows-mmlu-tat-qa-amazon.md?metric=Avg%20Relative%20Latency%20(vs%20Helium)>): Lower is better; unit: SCALAR.

- [Min Relative Latency (vs Helium)](<https://www.sota2.com/research/sota/llm-workflow-orchestration-on-representative-workflows-mmlu-tat-qa-amazon.md?metric=Min%20Relative%20Latency%20(vs%20Helium)>): Lower is better; unit: SCALAR.

- [Max Relative Latency (vs Helium)](<https://www.sota2.com/research/sota/llm-workflow-orchestration-on-representative-workflows-mmlu-tat-qa-amazon.md?metric=Max%20Relative%20Latency%20(vs%20Helium)>): Lower is better; unit: SCALAR.

## Recorded results

| Method | Avg Relative Latency (vs Helium) | Result date | Report paper |
| --- | --- | --- | --- |
| Helium | 1 | Not recorded | 69ba31aa0636e5bf4afc09a8 |
| OpWise | 1.25 | Not recorded | 69ba31aa0636e5bf4afc09a8 |
| LangGraph | 1.28 | Not recorded | 69ba31aa0636e5bf4afc09a8 |
| KVFlow | 1.32 | Not recorded | 69ba31aa0636e5bf4afc09a8 |
| Parrot | 1.44 | Not recorded | 69ba31aa0636e5bf4afc09a8 |
| AgentScope | 2.1 | Not recorded | 69ba31aa0636e5bf4afc09a8 |
| vLLM | 66.27 | Not recorded | 69ba31aa0636e5bf4afc09a8 |

### Helium

Avg Relative Latency (vs Helium): 1. Result ID: 69ba5d2d47f34ce924a41547.

[Reporting paper](<https://www.sota2.com/research/paper/69ba31aa0636e5bf4afc09a8.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69ba31aa0636e5bf4afc09a8.md>)

Recorded evaluation configuration: {"LLM Model":"Qwen3-8B","Hardware":"2x NVIDIA H100 NVL GPU","Sampling":"Greedy"}. This record does not assert independent reproduction.

### OpWise

Avg Relative Latency (vs Helium): 1.25. Result ID: 69ba5d2d47f34ce924a41542.

[Reporting paper](<https://www.sota2.com/research/paper/69ba31aa0636e5bf4afc09a8.md>)

Recorded evaluation configuration: {"LLM Model":"Qwen3-8B","Hardware":"2x NVIDIA H100 NVL GPU","Sampling":"Greedy"}. This record does not assert independent reproduction.

### LangGraph

Avg Relative Latency (vs Helium): 1.28. Result ID: 69ba5d2d47f34ce924a41543.

[Reporting paper](<https://www.sota2.com/research/paper/69ba31aa0636e5bf4afc09a8.md>)

Recorded evaluation configuration: {"LLM Model":"Qwen3-8B","Hardware":"2x NVIDIA H100 NVL GPU","Sampling":"Greedy"}. This record does not assert independent reproduction.

### KVFlow

Avg Relative Latency (vs Helium): 1.32. Result ID: 69ba5d2d47f34ce924a41546.

[Reporting paper](<https://www.sota2.com/research/paper/69ba31aa0636e5bf4afc09a8.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa79ed1e209131cea80099.md>)

Recorded evaluation configuration: {"LLM Model":"Qwen3-8B","Hardware":"2x NVIDIA H100 NVL GPU","Sampling":"Greedy"}. This record does not assert independent reproduction.

### Parrot

Avg Relative Latency (vs Helium): 1.44. Result ID: 69ba5d2d47f34ce924a41545.

[Reporting paper](<https://www.sota2.com/research/paper/69ba31aa0636e5bf4afc09a8.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73601e209131cea5c044.md>)

Recorded evaluation configuration: {"LLM Model":"Qwen3-8B","Hardware":"2x NVIDIA H100 NVL GPU","Sampling":"Greedy"}. This record does not assert independent reproduction.

### AgentScope

Avg Relative Latency (vs Helium): 2.1. Result ID: 69ba5d2d47f34ce924a41544.

[Reporting paper](<https://www.sota2.com/research/paper/69ba31aa0636e5bf4afc09a8.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73461e209131cea5b7ab.md>)

Recorded evaluation configuration: {"LLM Model":"Qwen3-8B","Hardware":"2x NVIDIA H100 NVL GPU","Sampling":"Greedy"}. This record does not assert independent reproduction.

### vLLM

Avg Relative Latency (vs Helium): 66.27. Result ID: 69ba5d2d47f34ce924a41541.

[Reporting paper](<https://www.sota2.com/research/paper/69ba31aa0636e5bf4afc09a8.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa708c1e209131cea4ba94.md>)

Recorded evaluation configuration: {"LLM Model":"Qwen3-8B","Hardware":"2x NVIDIA H100 NVL GPU","Sampling":"Greedy"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.