Tool Use and Reasoning on Earth-Bench
70.82AccuracyRS-Claw
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| RS-ClawModel=GPT-5, Evaluation Mode=IF2026.05 | 70.82 | — | 75.57 | 62.63 | — | — | |
| RS-ClawModel=GPT-5, Evaluation Mode=AP2026.05 | 68.67 | — | 73.1 | 59.52 | — | — | |
| OpenEarth-AgentLLM=GPT-5, Tool Set=Full Tools2026.03 | 67.61 | — | — | — | — | — | |
| GPT-5Regime=Auto-Planning (AP)2025.09 | 65.99 | 2.356 | 68.74 | 57.71 | 44.97 | 26.11 | |
| FlatModel=GPT-5, Evaluation Mode=AP2026.05 | 65.67 | — | 67.4 | 55.89 | — | — | |
| FlatModel=GPT-5, Evaluation Mode=IF2026.05 | 64.38 | — | 69.86 | 58.74 | — | — | |
| Earth-AgentLLM=GPT-52026.03 | 63.16 | — | — | — | — | — | |
| GPT-5Regime=Instruction-Following (IF)2025.09 | 62.35 | 2.9093 | 71.41 | 61.06 | 46.01 | 25.91 | |
| OpenEarth-AgentLLM=Seed-1.6, Tool Set=Full Tools2026.03 | 61.94 | — | — | — | — | — | |
| RAGModel=GPT-5, Evaluation Mode=IF2026.05 | 60.09 | — | 59.05 | 49.9 | — | — | |
| OpenEarth-AgentLLM=GPT-5, Tool Set=6 Tools2026.03 | 59.92 | — | — | — | — | — | |
| Seed-1.6Regime=Instruction-Following (IF)2025.09 | 59.51 | 1.3408 | 59.44 | 46.79 | 35.47 | 23.13 | |
| Earth-AgentLLM=Seed-1.62026.03 | 59.51 | — | — | — | — | — | |
| RAGModel=GPT-5, Evaluation Mode=AP2026.05 | 59.23 | — | 58.63 | 49.66 | — | — | |
| OpenEarth-AgentLLM=Kimik2, Tool Set=Full Tools2026.03 | 59.11 | — | — | — | — | — | |
| OpenEarth-AgentLLM=Gemini-2.5-Flash, Tool Set=Full Tools2026.03 | 57.89 | — | — | — | — | — | |
| RS-ClawModel=DeepSeek-V3.1, Evaluation Mode=AP2026.05 | 57.08 | — | 80.77 | 68.62 | — | — | |
| Kimik2Regime=Instruction-Following (IF)2025.09 | 56.68 | 2.1793 | 78.86 | 68.83 | 51.15 | 30.45 | |
| Earth-AgentLLM=Kimik22026.03 | 56.68 | — | — | — | — | — | |
| OpenEarth-AgentLLM=DeepSeek-V3.1, Tool Set=Full Tools2026.03 | 56.68 | — | — | — | — | — | |
| RS-ClawModel=DeepSeek-V3.1, Evaluation Mode=IF2026.05 | 55.79 | — | 81.17 | 69.44 | — | — | |
| Earth-AgentLLM=Gemini-2.5-Flash2026.03 | 55.06 | — | — | — | — | — | |
| OpenEarth-AgentLLM=Gemini-2.5-Flash, Tool Set=6 Tools2026.03 | 53.44 | — | — | — | — | — | |
| Gemini-2.5Regime=Instruction-Following (IF)2025.09 | 53.04 | 2.4595 | 61.63 | 50.72 | 41.04 | 23.43 | |
| OpenEarth-AgentLLM=Kimik2, Tool Set=6 Tools2026.03 | 52.63 | — | — | — | — | — | |
| OpenEarth-AgentLLM=Qwen3-Max, Tool Set=Full Tools2026.03 | 52.63 | — | — | — | — | — | |
| OpenEarth-AgentLLM=Seed-1.6, Tool Set=6 Tools2026.03 | 52.63 | — | — | — | — | — | |
| Seed-1.6Regime=Auto-Planning (AP)2025.09 | 52.48 | 1.311 | 55.43 | 40.67 | 28.39 | 18.32 | |
| Gemini-2.5Regime=Auto-Planning (AP)2025.09 | 52.23 | 2.9958 | 58.04 | 45.31 | 31.32 | 17.26 | |
| DeepSeek-V3.1Regime=Instruction-Following (IF)2025.09 | 52.23 | 2.6303 | 78.66 | 64.5 | 49.58 | 31.36 | |
| Earth-AgentLLM=DeepSeek-V3.12026.03 | 52.23 | — | — | — | — | — | |
| DeepSeek-V3.1Regime=Auto-Planning (AP)2025.09 | 51.42 | 2.6116 | 78.31 | 62.73 | 48.54 | 30.81 | |
| FlatModel=DeepSeek-V3.1, Evaluation Mode=IF2026.05 | 51.07 | — | 76.66 | 62.19 | — | — | |
| OpenEarth-AgentLLM=DeepSeek-V3.1, Tool Set=6 Tools2026.03 | 51.02 | — | — | — | — | — | |
| Kimik2Regime=Auto-Planning (AP)2025.09 | 50.61 | 1.8542 | 71.03 | 57.57 | 42.11 | 25.9 | |
| Qwen3-MaxRegime=Auto-Planning (AP)2025.09 | 50.2 | 1.881 | 69.56 | 53.28 | 37.02 | 21.83 | |
| FlatModel=DeepSeek-V3.1, Evaluation Mode=AP2026.05 | 49.36 | — | 77.25 | 61.19 | — | — | |
| Qwen3-MaxRegime=Instruction-Following (IF)2025.09 | 47.37 | 1.9511 | 70.14 | 56.02 | 42.74 | 26.27 | |
| Earth-AgentLLM=Qwen3-Max2026.03 | 47.37 | — | — | — | — | — | |
| OpenEarth-AgentLLM=Qwen3-Max, Tool Set=6 Tools2026.03 | 46.97 | — | — | — | — | — | |
| RAGModel=DeepSeek-V3.1, Evaluation Mode=IF2026.05 | 46.78 | — | 61.22 | 49.97 | — | — | |
| GPT-4oRegime=Instruction-Following (IF)2025.09 | 44.94 | 2.6007 | 67.02 | 53.04 | 47.41 | 27.92 | |
| LLaMA-4Regime=Auto-Planning (AP)2025.09 | 44.94 | 0.2886 | 16.51 | 2.45 | 1.7 | 1.3 | |
| GPT-4oRegime=Auto-Planning (AP)2025.09 | 43.72 | 2.1211 | 65.65 | 50.76 | 46.26 | 26.55 | |
| Qwen-PlusRegime=Auto-Planning (AP)2025.09 | 42.51 | 1.5119 | 52.04 | 30.75 | 11.69 | 9.12 | |
| RAGModel=DeepSeek-V3.1, Evaluation Mode=AP2026.05 | 39.91 | — | 62.39 | 47.82 | — | — | |
| LLaMA-4Regime=Instruction-Following (IF)2025.09 | 38.46 | 0.3211 | 22.41 | 12.05 | 9.05 | 6.46 | |
| Qwen-PlusRegime=Instruction-Following (IF)2025.09 | 38.46 | 1.5854 | 55.96 | 39.77 | 25.51 | 16.95 | |
| GLM-4.5vRegime=Instruction-Following (IF)2025.09 | 35.25 | 2.0129 | 46.69 | 35.24 | 19.95 | 15.37 | |
| RS-ClawModel=Qwen3-32b, Evaluation Mode=AP2026.05 | 33.05 | — | 50.22 | 32.06 | — | — | |
| GLM-4.5vRegime=Auto-Planning (AP)2025.09 | 32.86 | 1.7123 | 42.48 | 28.57 | 14.12 | 11.02 | |
| MistralRegime=Auto-Planning (AP)2025.09 | 29.96 | 0.9552 | 27.73 | 11.78 | 9.13 | 7.24 | |
| RS-ClawModel=Qwen3-32b, Evaluation Mode=IF2026.05 | 29.18 | — | 60.77 | 44.91 | — | — | |
| InternVL-3.5Regime=Auto-Planning (AP)2025.09 | 26.72 | 0.1206 | 8.83 | 3.87 | 2.02 | 1.46 | |
| InternVL-3.5Regime=Instruction-Following (IF)2025.09 | 26.72 | 0.175 | 16.62 | 10.59 | 9.32 | 5.32 | |
| FlatModel=Qwen3-32b, Evaluation Mode=IF2026.05 | 25.32 | — | 42.36 | 34.31 | — | — | |
| Qwen3-32BRegime=Instruction-Following (IF)2025.09 | 24.8 | 1.901 | 42.39 | 33.79 | 26.1 | 17.73 | |
| RAGModel=Qwen3-32b, Evaluation Mode=IF2026.05 | 22.75 | — | 35.36 | 26.06 | — | — | |
| MistralRegime=Instruction-Following (IF)2025.09 | 22.67 | 0.8802 | 29.64 | 20.9 | 18.13 | 11.66 | |
| Qwen3-32BRegime=Auto-Planning (AP)2025.09 | 20.65 | 2.7274 | 39.76 | 21.56 | 9.51 | 8.17 | |
| FlatModel=Qwen3-32b, Evaluation Mode=AP2026.05 | 20.6 | — | 39.86 | 22.14 | — | — | |
| RAGModel=Qwen3-32b, Evaluation Mode=AP2026.05 | 20.17 | — | 30.04 | 20.75 | — | — |