Autonomous Agent Spatio-Temporal Reasoning on STT-Arena
41.78Pass@1 (Easy)Claude-4.6-Opus
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Claude-4.6-OpusModel Category=Closed-Source LLMs2026.05 | 41.78 | 31.34 | 23.73 | 52.22 | 35.39 | |
| GPT-5.4Model Category=Closed-Source LLMs2026.05 | 39.91 | 29.85 | 24.29 | 52.22 | 34.51 | |
| Gemini-3.1-ProModel Category=Closed-Source LLMs2026.05 | 37.56 | 30.35 | 21.47 | 52.22 | 33.19 | |
| Deepseek-V3.2Model Category=Open-source LLMs2026.05 | 36.62 | 27.86 | 15.82 | 63.33 | 32.16 | |
| Qwen-3.6-PlusModel Category=Closed-Source LLMs2026.05 | 35.21 | 24.38 | 23.16 | 54.44 | 31.42 | |
| Claude-4.6-SonnetModel Category=Closed-Source LLMs2026.05 | 35.21 | 29.85 | 19.77 | 58.89 | 32.74 | |
| GLM-5.1Model Category=Open-source LLMs2026.05 | 35.21 | 23.38 | 15.25 | 41.11 | 27.31 | |
| Qwen-3.5-397B-A17BModel Category=Open-source LLMs2026.05 | 33.8 | 21.89 | 17.51 | 54.45 | 28.78 | |
| Deepseek-V4-ProModel Category=Open-source LLMs2026.05 | 32.86 | 20.4 | 15.25 | 45.56 | 26.29 | |
| GLM-5Model Category=Open-source LLMs2026.05 | 30.05 | 21.39 | 10.73 | 43.33 | 24.23 | |
| STT-Agent-4B (SFT+RL)Model Category=STT-Agent, Training Protocol=SFT+RL2026.05 | 29.11 | 19.9 | 14.12 | 64.44 | 27.17 | |
| Gemini-2.5-flashModel Category=More Efficient LLMs2026.05 | 28.17 | 13.43 | 12.43 | 31.11 | 20.11 | |
| Qwen-3.5-9BModel Category=More Efficient LLMs2026.05 | 27.23 | 12.93 | 11.3 | 32.22 | 19.53 | |
| MiniMax-M2.5Model Category=Open-source LLMs2026.05 | 26.76 | 15.92 | 12.43 | 42.22 | 21.88 | |
| STT-Agent-4B (SFT)Model Category=STT-Agent, Training Protocol=SFT2026.05 | 26.76 | 17.41 | 13.56 | 61.11 | 25.11 | |
| MiniMax-M2.7Model Category=Open-source LLMs2026.05 | 26.29 | 15.92 | 13.56 | 36.67 | 21.29 | |
| Kimi-K2.5Model Category=Open-source LLMs2026.05 | 25.35 | 13.43 | 14.12 | 37.78 | 19.18 | |
| Llama-3.3-70BModel Category=Open-source LLMs2026.05 | 22.07 | 11.44 | 11.3 | 30 | 17.18 | |
| Qwen-3.5-35B-A3BModel Category=More Efficient LLMs2026.05 | 19.72 | 13.43 | 7.34 | 17.78 | 12.48 | |
| GPT-5.4-miniModel Category=More Efficient LLMs2026.05 | 19.72 | 13.43 | 7.34 | 6.67 | 12.92 | |
| Qwen-3-4B (baseline)Model Category=STT-Agent2026.05 | 18.31 | 9.46 | 2.82 | 10 | 10.57 | |
| Qwen-3-8BModel Category=More Efficient LLMs2026.05 | 16.9 | 13.43 | 11.3 | 20 | 14.83 | |
| Llama-3.1-8BModel Category=More Efficient LLMs2026.05 | 7.98 | 3.98 | 2.82 | 5.56 | 5.14 |