Rule-based Verification of Environment Simulation on Rule-based Verification Benchmark
82.9MCPGPT-5.4
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| GPT-5.4Model Category=Frontier2026.06 | 82.9 | 66.95 | 68.13 | 62.37 | 81.08 | 81.61 | 67.46 | 72.93 | |
| Claude Opus 4.6Model Category=Frontier2026.06 | 76.9 | 77.55 | 68.6 | 54.4 | 80.06 | 75.71 | 61.81 | 70.72 | |
| Claude Sonnet 4.6Model Category=Frontier2026.06 | 76.43 | 71.9 | 64.43 | 55.27 | 75.23 | 77.16 | 52.57 | 67.57 | |
| Qwen3.6-Max-PreviewModel Category=Qwen2026.06 | 73.4 | 64.05 | 59.37 | 45.03 | 60.19 | 67.02 | 50.47 | 59.93 | |
| GLM-5.1Model Category=Open-weight2026.06 | 72.53 | 71.7 | 55.47 | 41.77 | 67.98 | 72.06 | 55.98 | 62.5 | |
| Qwen-AgentWorld-397B-A17BModel Category=Ours2026.06 | 72.37 | 58.95 | 62.63 | 60.33 | 76.79 | 80.74 | 58.01 | 67.12 | |
| Kimi K2.6Model Category=Open-weight2026.06 | 70.3 | 69.4 | 55.53 | 45.2 | 61.9 | 75.63 | 62.49 | 62.92 | |
| DeepSeek-V4-ProModel Category=Open-weight2026.06 | 69.9 | 56.85 | 57.4 | 44.5 | 76.36 | 72.83 | 61.28 | 62.73 | |
| Gemini 3.1 ProModel Category=Frontier2026.06 | 68.77 | 61.45 | 62.33 | 53.47 | 76.73 | 75.32 | 65.5 | 66.22 | |
| Qwen3.6-PlusModel Category=Qwen2026.06 | 67.57 | 67.9 | 58.43 | 47.57 | 63.81 | 69.85 | 54.29 | 61.35 | |
| Qwen3.5-397B-A17BModel Category=Ours2026.06 | 67.33 | 63.5 | 51.8 | 44.17 | 69.25 | 70.22 | 54.03 | 60.04 | |
| Qwen-AgentWorld-35B-A3BModel Category=Ours2026.06 | 60.47 | 44.9 | 54.53 | 48.73 | 62.34 | 72.37 | 57.61 | 57.28 | |
| Qwen3.5-35B-A3BModel Category=Ours2026.06 | 57.8 | 40.7 | 36.83 | 28.5 | 61.1 | 68.49 | 44.07 | 48.21 | |
| MiniMax-M2.7Model Category=Open-weight2026.06 | 55.03 | 44.3 | 32.4 | 25.17 | 71.87 | 70.36 | 49.39 | 49.79 | |
| Qwen3.6-35B-A3BModel Category=Qwen2026.06 | 49.5 | 47 | 41.3 | 35.07 | 59.79 | 67.83 | 44.03 | 49.22 |