Long-horizon agentic task on HLE
60PerformanceAggAgent
Evaluation Results
| Method | Links | |
|---|---|---|
| AggAgentBackbone=MiniMax-M2.5, Aggregation Category=LLM-based Aggregation, Number of Rollouts (K)=82026.04 | 60 | |
| Best-of-NBackbone=MiniMax-M2.5, Aggregation Category=Heuristic Aggregation, Number of Rollouts (K)=82026.04 | 58.06 | |
| Summary AggregationBackbone=MiniMax-M2.5, Aggregation Category=LLM-based Aggregation, Number of Rollouts (K)=82026.04 | 58.06 | |
| Weighted VotingBackbone=MiniMax-M2.5, Aggregation Category=Heuristic Aggregation, Number of Rollouts (K)=82026.04 | 57.42 | |
| AggAgentBackbone=Qwen3.5-122B, Aggregation Category=LLM-based Aggregation, Number of Rollouts (K)=82026.04 | 54.19 | |
| Solution AggregationBackbone=MiniMax-M2.5, Aggregation Category=LLM-based Aggregation, Number of Rollouts (K)=82026.04 | 54.19 | |
| Best-of-NBackbone=Qwen3.5-122B, Aggregation Category=Heuristic Aggregation, Number of Rollouts (K)=82026.04 | 53.55 | |
| Majority VotingBackbone=MiniMax-M2.5, Aggregation Category=Heuristic Aggregation, Number of Rollouts (K)=82026.04 | 53.55 | |
| Solution AggregationBackbone=Qwen3.5-122B, Aggregation Category=LLM-based Aggregation, Number of Rollouts (K)=82026.04 | 50.97 | |
| Summary AggregationBackbone=Qwen3.5-122B, Aggregation Category=LLM-based Aggregation, Number of Rollouts (K)=82026.04 | 50.32 | |
| Kimi-K2.5-1TCM=w/o2026.05 | 50.2 | |
| Fewest Tool CallsBackbone=MiniMax-M2.5, Aggregation Category=Heuristic Aggregation, Number of Rollouts (K)=82026.04 | 46.45 | |
| Pass@1Backbone=MiniMax-M2.5, Aggregation Category=Single Rollout, Number of Rollouts (K)=12026.04 | 45.73 | |
| Weighted VotingBackbone=Qwen3.5-122B, Aggregation Category=Heuristic Aggregation, Number of Rollouts (K)=82026.04 | 45.16 | |
| Majority VotingBackbone=Qwen3.5-122B, Aggregation Category=Heuristic Aggregation, Number of Rollouts (K)=82026.04 | 43.87 | |
| Claude-4.5-OpusCM=w/o2026.05 | 43.4 | |
| Fewest Tool CallsBackbone=Qwen3.5-122B, Aggregation Category=Heuristic Aggregation, Number of Rollouts (K)=82026.04 | 41.29 | |
| Pass@1Backbone=Qwen3.5-122B, Aggregation Category=Single Rollout, Number of Rollouts (K)=12026.04 | 39.35 | |
| SAMBackbone=GLM-4.72026.05 | 38.2 | |
| GLM-4.7CM=summary2026.05 | 37.5 | |
| AggAgentBackbone=GLM-4.7-Flash, Aggregation Category=LLM-based Aggregation, Number of Rollouts (K)=82026.04 | 37.42 | |
| GLM-4.7CM=w/o2026.05 | 37.2 | |
| GLM-4.7CM=recent-k2026.05 | 37.2 | |
| SAMBackbone=Qwen3.5-35B-A3B2026.05 | 37.2 | |
| GLM-4.7CM=discard-tool2026.05 | 36.5 | |
| Qwen3.5-35B-A3BCM=discard-tool2026.05 | 36 | |
| GPT-5.4CM=w/o2026.05 | 35.2 | |
| Qwen3.5-35B-A3BCM=summary2026.05 | 35.2 | |
| Summary AggregationBackbone=GLM-4.7-Flash, Aggregation Category=LLM-based Aggregation, Number of Rollouts (K)=82026.04 | 34.84 | |
| Qwen3.5-35B-A3BCM=recent-k2026.05 | 34.2 | |
| Qwen3.5-35B-A3BCM=w/o2026.05 | 34 | |
| Solution AggregationBackbone=GLM-4.7-Flash, Aggregation Category=LLM-based Aggregation, Number of Rollouts (K)=82026.04 | 32.9 | |
| Best-of-NBackbone=GLM-4.7-Flash, Aggregation Category=Heuristic Aggregation, Number of Rollouts (K)=82026.04 | 30.97 | |
| IterResearcher-30B-A3BCategory=Agent systems2026.05 | 28.8 | |
| Weighted VotingBackbone=GLM-4.7-Flash, Aggregation Category=Heuristic Aggregation, Number of Rollouts (K)=82026.04 | 27.74 | |
| Fewest Tool CallsBackbone=GLM-4.7-Flash, Aggregation Category=Heuristic Aggregation, Number of Rollouts (K)=82026.04 | 27.1 | |
| Majority VotingBackbone=GLM-4.7-Flash, Aggregation Category=Heuristic Aggregation, Number of Rollouts (K)=82026.04 | 26.45 | |
| Pass@1Backbone=GLM-4.7-Flash, Aggregation Category=Single Rollout, Number of Rollouts (K)=12026.04 | 25 | |
| OpenAI-o3CM=w/o2026.05 | 24.9 | |
| WebThinker-32BCategory=Agent systems2026.05 | 15.8 | |
| WebSailor-32BCategory=Agent systems2026.05 | 9.6 |