Agent on Terminal-Bench
45AccuracyDeepSeek V3.2
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| DeepSeek V3.2Evaluation Mode=Chat2025.12 | 45 | — | |
| LongCat-Flash Exp-ChatEvaluation Mode=Chat2025.12 | 42.5 | — | |
| GLM 4.6Evaluation Mode=Chat2025.12 | 40.5 | — | |
| LongCat-Flash ChatEvaluation Mode=Chat2025.12 | 39.5 | — | |
| ReflectionInference Method=Best-of-N (pass@5), Temperature=0.92026.05 | 25.8 | — | |
| LiTS-FactInference Method=Best-of-N (pass@5), Temperature=0.92026.05 | 25.8 | — | |
| No MemoryInference Method=Best-of-N (pass@5), Temperature=0.92026.05 | 23.6 | — | |
| PromptBridgeModel Role=Target Model, Model=GPT-4o, Source Model=o32025.12 | 18.75 | 25 | |
| Direct TransferModel Role=Target Model, Model=GPT-4o, Source Model=o32025.12 | 15 | — | |
| PromptBridgeModel Role=Target Model, Model=Llama-3.1-70B-Instruct, Source Model=o32025.12 | 8.75 | 40 | |
| ReActInference Method=Single Trajectory, Temperature=02026.05 | 7.9 | — | |
| Direct TransferModel Role=Target Model, Model=Llama-3.1-70B-Instruct, Source Model=o32025.12 | 6.25 | — |