Environment Synthesis on Programming-based Environments
1,000Environment CountAWM
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| AWMSynthetic Generation=true, Reliance=Names Only, SQL=true2026.02 | 1,000 | 35.1 | 1,984.7 | |
| EnvScalerSynthetic Generation=true, Reliance=Task Set, SQL=false2026.02 | 191 | 18.6 | 662.1 | |
| MCP-UniverseSynthetic Generation=false, Reliance=Real APIs2026.02 | 11 | 12.1 | — | |
| AutoForgeSynthetic Generation=true, Reliance=Tool Doc, SQL=false2026.02 | 10 | — | — | |
| τ²-benchSynthetic Generation=false, Reliance=Human, SQL=false2026.02 | 3 | 22.7 | — | |
| τ-benchSynthetic Generation=false, Reliance=Human, SQL=false2026.02 | 2 | 12.5 | — |