Agentic Terminal Tasks on Terminal-Bench out-of-distribution 2.0 (test)
39.4Avg@5Tree-GRPO
Evaluation Results
| Method | Links | |
|---|---|---|
| Tree-GRPOBase Model=Qwen3.5-35B-A3B2026.05 | 39.4 | |
| Tree-GRPOBase Model=Nemotron-3-Super-120B-A12B2026.05 | 37.2 | |
| Flat GRPOBase Model=Qwen3.5-35B-A3B2026.05 | 34.2 | |
| Flat GRPOBase Model=Nemotron-3-Super-120B-A12B2026.05 | 33.8 | |
| BaseBase Model=Nemotron-3-Super-120B-A12B2026.05 | 30.3 | |
| BaseBase Model=Qwen3.5-35B-A3B2026.05 | 26.1 |