Software Engineering Reasoning on Terminal-Bench (TB2) 2.0
36Resolution RateTACT-GATE
Evaluation results
| Method | Links | ||
|---|---|---|---|
| TACT-GATEBackbone=Qwen3.5-27B2026.05 | 36 | 41.2 | |
| TACT-ADDBackbone=Qwen3.5-27B2026.05 | 35.9 | 42.7 | |
| AGENTDEBUGBackbone=Qwen3.5-27B, rollouts=at least two per task2026.05 | 32.8 | 81.7 | |
| TACT-CAPBackbone=Qwen3.5-27B2026.05 | 32.6 | 36.5 | |
| AGENTREFLECTBackbone=Qwen3.5-27B2026.05 | 32 | 46.8 | |
| RECAPBackbone=Qwen3.5-27B2026.05 | 31.1 | 49.5 | |
| Base ModelBackbone=Qwen3.5-27B2026.05 | 30.3 | 48.2 | |
| REMINDERBackbone=Qwen3.5-27B2026.05 | 29.8 | 50.4 | |
| TACT-GATEBackbone=Gemma-4-26B-A4B-it2026.05 | 19.1 | 16 | |
| TACT-ADDBackbone=Gemma-4-26B-A4B-it2026.05 | 18.9 | 16.5 | |
| AGENTDEBUGBackbone=Gemma-4-26B-A4B-it, rollouts=at least two per task2026.05 | 17.2 | 31.5 | |
| TACT-CAPBackbone=Gemma-4-26B-A4B-it2026.05 | 17 | 13.9 | |
| RECAPBackbone=Gemma-4-26B-A4B-it2026.05 | 16.7 | 18.4 | |
| AGENTREFLECTBackbone=Gemma-4-26B-A4B-it2026.05 | 16 | 17.7 | |
| REMINDERBackbone=Gemma-4-26B-A4B-it2026.05 | 15.9 | 18.6 | |
| Base ModelBackbone=Gemma-4-26B-A4B-it2026.05 | 15.7 | 18.2 |