Tool-use evaluation on OpenEarthAgent Step-by-step evaluation tool-agnostic rollouts
0.9951Instruction AdherenceOpenEarthAgent
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| OpenEarthAgentParameters=4B2026.02 | 0.9951 | 0.9718 | 0.9608 | 0.621 | 0.8364 | |
| gpt-4o2026.02 | 0.9918 | 0.9388 | 0.8548 | 0.458 | 0.8676 | |
| Qwen3-Instruct-2507Parameters=4B2026.02 | 0.9734 | 0.8694 | 0.8412 | 0.3355 | 0.8328 | |
| Qwen2.5-InstructParameters=7B2026.02 | 0.9408 | 0.8551 | 0.7846 | 0.38 | 0.8008 | |
| Qwen2.5-InstructParameters=3B2026.02 | 0.8507 | 0.7213 | 0.6487 | 0.2412 | 0.6875 | |
| o4-mini2026.02 | 0.8368 | 0.6833 | 0.6417 | 0.3795 | 0.8948 | |
| Mistralv0.3-InstructParameters=7B2026.02 | 0.6414 | 0.3512 | 0.2611 | 0.1245 | 0.2404 | |
| Llama-3.1-InstructParameters=8B2026.02 | 0.4707 | 0.393 | 0.3411 | 0.1749 | 0.7908 | |
| Internlm3-InstructParameters=8B2026.02 | 0.4454 | 0.3846 | 0.2782 | 0.1325 | 0.2984 |