# Inference Time Consumption on GAIA

Source: [Inference Time Consumption on GAIA](<https://www.sota2.com/research/sota/inference-time-consumption-on-gaia>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-26T05:48:26.967Z

Evaluation of model inference latency consumption across various operational categories (Research And Data, Browser Automation, File Systems, Developer Tools) using the GAIA dataset.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Inference Time Consumption |
| Dataset (including any stated split) | GAIA |
| Metric | Latency (Research And Data) |
| Unit | TIME |
| Direction | Lower is better |
| Results loaded | 16 of 16 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Latency (Research And Data)](<https://www.sota2.com/research/sota/inference-time-consumption-on-gaia.md?metric=Latency%20(Research%20And%20Data)>): Lower is better; unit: TIME.

- [Latency (Browser Automation)](<https://www.sota2.com/research/sota/inference-time-consumption-on-gaia.md?metric=Latency%20(Browser%20Automation)>): Lower is better; unit: TIME.

- [Latency (File Systems)](<https://www.sota2.com/research/sota/inference-time-consumption-on-gaia.md?metric=Latency%20(File%20Systems)>): Lower is better; unit: TIME.

- [Latency (Developer Tools)](<https://www.sota2.com/research/sota/inference-time-consumption-on-gaia.md?metric=Latency%20(Developer%20Tools)>): Lower is better; unit: TIME.

- [Average Latency](<https://www.sota2.com/research/sota/inference-time-consumption-on-gaia.md?metric=Average%20Latency>): Lower is better; unit: TIME.

## Recorded results

| Method | Latency (Research And Data) | Result date | Report paper |
| --- | --- | --- | --- |
| LeechHijack | 11.2 | Not recorded | 692fc3fcd6b0dd3342933209 |
| Normal | 12.7 | Not recorded | 692fc3fcd6b0dd3342933209 |
| Normal | 22.3 | Not recorded | 692fc3fcd6b0dd3342933209 |
| Normal | 24.2 | Not recorded | 692fc3fcd6b0dd3342933209 |
| LeechHijack | 28.6 | Not recorded | 692fc3fcd6b0dd3342933209 |
| LeechHijack | 31.8 | Not recorded | 692fc3fcd6b0dd3342933209 |
| Hybrid (Pydantic-AI) | 73.8 | Not recorded | 692fc3fcd6b0dd3342933209 |
| Hybrid (Pydantic-AI) | 98.2 | Not recorded | 692fc3fcd6b0dd3342933209 |
| Localized (OpenManus) | 118.9 | Not recorded | 692fc3fcd6b0dd3342933209 |
| Normal | 118.9 | Not recorded | 692fc3fcd6b0dd3342933209 |
| Cloud-Hosted (OpenManus) | 127.8 | Not recorded | 692fc3fcd6b0dd3342933209 |
| Cloud-Hosted (OpenManus) | 128.6 | Not recorded | 692fc3fcd6b0dd3342933209 |
| Localized (OpenManus) | 138.1 | Not recorded | 692fc3fcd6b0dd3342933209 |
| LeechHijack | 138.1 | Not recorded | 692fc3fcd6b0dd3342933209 |
| Hybrid (OWL) | 227.6 | Not recorded | 692fc3fcd6b0dd3342933209 |
| Hybrid (OWL) | 279.3 | Not recorded | 692fc3fcd6b0dd3342933209 |

### LeechHijack

Latency (Research And Data): 11.2. Result ID: 699709d55698665ec4573bd6.

[Reporting paper](<https://www.sota2.com/research/paper/692fc3fcd6b0dd3342933209.md>)

[Method origin paper](<https://www.sota2.com/research/paper/692fc3fcd6b0dd3342933209.md>)

Recorded evaluation configuration: {"Model":"Qwen"}. This record does not assert independent reproduction.

### Normal

Latency (Research And Data): 12.7. Result ID: 699709d55698665ec4573bd5.

[Reporting paper](<https://www.sota2.com/research/paper/692fc3fcd6b0dd3342933209.md>)

Recorded evaluation configuration: {"Model":"Qwen"}. This record does not assert independent reproduction.

### Normal

Latency (Research And Data): 22.3. Result ID: 699709d55698665ec4573bd7.

[Reporting paper](<https://www.sota2.com/research/paper/692fc3fcd6b0dd3342933209.md>)

Recorded evaluation configuration: {"Model":"GPT"}. This record does not assert independent reproduction.

### Normal

Latency (Research And Data): 24.2. Result ID: 699709d55698665ec4573bd9.

[Reporting paper](<https://www.sota2.com/research/paper/692fc3fcd6b0dd3342933209.md>)

Recorded evaluation configuration: {"Model":"Gemini"}. This record does not assert independent reproduction.

### LeechHijack

Latency (Research And Data): 28.6. Result ID: 699709d55698665ec4573bda.

[Reporting paper](<https://www.sota2.com/research/paper/692fc3fcd6b0dd3342933209.md>)

[Method origin paper](<https://www.sota2.com/research/paper/692fc3fcd6b0dd3342933209.md>)

Recorded evaluation configuration: {"Model":"Gemini"}. This record does not assert independent reproduction.

### LeechHijack

Latency (Research And Data): 31.8. Result ID: 699709d55698665ec4573bd8.

[Reporting paper](<https://www.sota2.com/research/paper/692fc3fcd6b0dd3342933209.md>)

[Method origin paper](<https://www.sota2.com/research/paper/692fc3fcd6b0dd3342933209.md>)

Recorded evaluation configuration: {"Model":"GPT"}. This record does not assert independent reproduction.

### Hybrid (Pydantic-AI)

Latency (Research And Data): 73.8. Result ID: 699709d55698665ec4573bc5.

[Reporting paper](<https://www.sota2.com/research/paper/692fc3fcd6b0dd3342933209.md>)

[Method origin paper](<https://www.sota2.com/research/paper/692fc3fcd6b0dd3342933209.md>)

Recorded evaluation configuration: {"MCP Tool":"Normal"}. This record does not assert independent reproduction.

### Hybrid (Pydantic-AI)

Latency (Research And Data): 98.2. Result ID: 699709d55698665ec4573bc6.

[Reporting paper](<https://www.sota2.com/research/paper/692fc3fcd6b0dd3342933209.md>)

[Method origin paper](<https://www.sota2.com/research/paper/692fc3fcd6b0dd3342933209.md>)

Recorded evaluation configuration: {"MCP Tool":"LeechHijack"}. This record does not assert independent reproduction.

### Localized (OpenManus)

Latency (Research And Data): 118.9. Result ID: 699709d55698665ec4573bc3.

[Reporting paper](<https://www.sota2.com/research/paper/692fc3fcd6b0dd3342933209.md>)

[Method origin paper](<https://www.sota2.com/research/paper/692fc3fcd6b0dd3342933209.md>)

Recorded evaluation configuration: {"MCP Tool":"Normal"}. This record does not assert independent reproduction.

### Normal

Latency (Research And Data): 118.9. Result ID: 699709d55698665ec4573bd3.

[Reporting paper](<https://www.sota2.com/research/paper/692fc3fcd6b0dd3342933209.md>)

Recorded evaluation configuration: {"Model":"Deepseek"}. This record does not assert independent reproduction.

### Cloud-Hosted (OpenManus)

Latency (Research And Data): 127.8. Result ID: 699709d55698665ec4573bca.

[Reporting paper](<https://www.sota2.com/research/paper/692fc3fcd6b0dd3342933209.md>)

[Method origin paper](<https://www.sota2.com/research/paper/692fc3fcd6b0dd3342933209.md>)

Recorded evaluation configuration: {"MCP Tool":"LeechHijack"}. This record does not assert independent reproduction.

### Cloud-Hosted (OpenManus)

Latency (Research And Data): 128.6. Result ID: 699709d55698665ec4573bc9.

[Reporting paper](<https://www.sota2.com/research/paper/692fc3fcd6b0dd3342933209.md>)

[Method origin paper](<https://www.sota2.com/research/paper/692fc3fcd6b0dd3342933209.md>)

Recorded evaluation configuration: {"MCP Tool":"Normal"}. This record does not assert independent reproduction.

### Localized (OpenManus)

Latency (Research And Data): 138.1. Result ID: 699709d55698665ec4573bc4.

[Reporting paper](<https://www.sota2.com/research/paper/692fc3fcd6b0dd3342933209.md>)

[Method origin paper](<https://www.sota2.com/research/paper/692fc3fcd6b0dd3342933209.md>)

Recorded evaluation configuration: {"MCP Tool":"LeechHijack"}. This record does not assert independent reproduction.

### LeechHijack

Latency (Research And Data): 138.1. Result ID: 699709d55698665ec4573bd4.

[Reporting paper](<https://www.sota2.com/research/paper/692fc3fcd6b0dd3342933209.md>)

[Method origin paper](<https://www.sota2.com/research/paper/692fc3fcd6b0dd3342933209.md>)

Recorded evaluation configuration: {"Model":"Deepseek"}. This record does not assert independent reproduction.

### Hybrid (OWL)

Latency (Research And Data): 227.6. Result ID: 699709d55698665ec4573bc8.

[Reporting paper](<https://www.sota2.com/research/paper/692fc3fcd6b0dd3342933209.md>)

[Method origin paper](<https://www.sota2.com/research/paper/692fc3fcd6b0dd3342933209.md>)

Recorded evaluation configuration: {"MCP Tool":"LeechHijack"}. This record does not assert independent reproduction.

### Hybrid (OWL)

Latency (Research And Data): 279.3. Result ID: 699709d55698665ec4573bc7.

[Reporting paper](<https://www.sota2.com/research/paper/692fc3fcd6b0dd3342933209.md>)

[Method origin paper](<https://www.sota2.com/research/paper/692fc3fcd6b0dd3342933209.md>)

Recorded evaluation configuration: {"MCP Tool":"Normal"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
