Reasoning on GAIA text
69.9Average AccuracyWebAggregator
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| WebAggregatorSetting=Fine-tuned Foundations, Backbone=Qwen2.5-32B, Evaluation Protocol=Pass@32025.10 | 69.9 | 79.5 | 67.3 | 50 | — | — | |
| WebAggregatorSetting=Fine-tuned Foundations, Backbone=Qwen3-32B, Evaluation Protocol=Pass@32025.10 | 69.9 | 79.5 | 67.3 | 50 | — | — | |
| WebAggregatorSetting=Fine-tuned Foundations, Backbone=Qwen2.5-7B, Evaluation Protocol=Pass@32025.10 | 63.1 | 74.4 | 63.5 | 25 | — | — | |
| WebAggregatorSetting=Fine-tuned Foundations, Backbone=Qwen3-8B, Evaluation Protocol=Pass@32025.10 | 62.1 | 82.1 | 53.8 | 33.3 | — | — | |
| GPT-5Setting=Zero-shot Foundations2025.10 | 60.2 | 71.8 | 57.7 | 33.3 | — | — | |
| Claude-3.7-sonnetSetting=Zero-shot Foundations2025.10 | 60.2 | 74.4 | 55.8 | 33.3 | — | — | |
| WebAggregatorSetting=Fine-tuned Foundations, Backbone=Qwen3-32B, Evaluation Protocol=Pass@12025.10 | 56.3 | 69.2 | 55.8 | 16.7 | — | — | |
| WebSailorSetting=Fine-tuned Foundations, Backbone=Qwen2.5-32B2025.10 | 53.2 | — | — | — | — | — | |
| WebShaperSetting=Fine-tuned Foundations, Backbone=Qwen2.5-32B2025.10 | 52.2 | 61.5 | 53.8 | 16.7 | — | — | |
| WebAggregatorSetting=Fine-tuned Foundations, Backbone=Qwen2.5-32B, Evaluation Protocol=Pass@12025.10 | 51.5 | 66.7 | 44.2 | 33.3 | — | — | |
| WebThinkerSetting=Fine-tuned Foundations, Backbone=Qwen2.5-32B2025.10 | 48.5 | 56.4 | 50 | 16.7 | — | — | |
| GPT-4.1Setting=Zero-shot Foundations2025.10 | 43.7 | 51.3 | 44.2 | 16.7 | — | — | |
| CogKernal-ProSetting=Fine-tuned Foundations, Backbone=Qwen3-8B2025.10 | 43.7 | 56.4 | 42.3 | 8.3 | — | — | |
| WebAggregatorSetting=Fine-tuned Foundations, Backbone=Qwen3-8B, Evaluation Protocol=Pass@12025.10 | 42.7 | 61.5 | 34.6 | 16.7 | — | — | |
| Qwen3-32BSetting=Zero-shot Foundations2025.10 | 40.8 | 48.7 | 40.4 | 16.7 | — | — | |
| WebAggregatorSetting=Fine-tuned Foundations, Backbone=Qwen2.5-7B, Evaluation Protocol=Pass@12025.10 | 40.8 | 53.8 | 30.8 | 16.7 | — | — | |
| WebDancerSetting=Fine-tuned Foundations, Backbone=Qwen2.5-32B2025.10 | 40.7 | 46.1 | 44.2 | 8.3 | — | — | |
| WebSailorSetting=Fine-tuned Foundations, Backbone=Qwen2.5-7B2025.10 | 37.9 | — | — | — | — | — | |
| WebDancerSetting=Fine-tuned Foundations, Backbone=Qwen2.5-7B2025.10 | 31 | 41 | 30.7 | 0 | — | — | |
| Qwen2.5-32BSetting=Zero-shot Foundations2025.10 | 28.2 | 46.1 | 21.2 | 0 | — | — | |
| Claude-3.7-sonnetSetting=Non-Agentic2025.10 | 22.3 | 35.9 | 17.3 | 0 | — | — | |
| Qwen3-8BSetting=Zero-shot Foundations2025.10 | 18.4 | 33.3 | 11.5 | 0 | — | — | |
| Qwen2.5-7BSetting=Zero-shot Foundations2025.10 | 16.5 | 23.1 | 15.4 | 0 | — | — | |
| Qwen2.5-32BSetting=Non-Agentic2025.10 | 13.6 | 20.5 | 9.6 | 8.3 | — | — | |
| GPT-4.1Setting=Non-Agentic2025.10 | 11.7 | 10.3 | 13.5 | 8.3 | — | — | |
| Qwen3-32BSetting=Non-Agentic2025.10 | 8.7 | 17.9 | 3.8 | 0 | — | — | |
| Qwen2.5-7BSetting=Non-Agentic2025.10 | 6.8 | 12.8 | 3.8 | 0 | — | — | |
| Qwen3-8BSetting=Non-Agentic2025.10 | 6.8 | 12.8 | 3.8 | 0 | — | — |