Behavioral Similarity Analysis on τ-Bench and τ2-Bench (test)
82.6GED ScoreSonnet 4.5
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Sonnet 4.5Family=ANTHROPIC, Thinking Mode=no-think2026.04 | 82.6 | 3.14 | 37.1 | 95.1 | 79.1 | 79.9 | 94 | 84.3 | 4.07 | 3.89 | 3.66 | 3.87 | |
| Opus 4.1Family=ANTHROPIC, Thinking Mode=thinking2026.04 | 79.7 | 3.25 | 35.5 | 94.6 | 81 | 74.4 | 93.7 | 83 | 4.02 | 3.8 | 3.72 | 3.85 | |
| Kimi-K2Family=MOONSHOT, Thinking Mode=thinking2026.04 | 78.1 | 2.81 | 34.3 | 93.8 | 82.6 | 70.8 | 94.7 | 82.7 | 3.86 | 3.57 | 3.51 | 3.65 | |
| Gemini 3 ProFamily=GOOGLE2026.04 | 77.5 | 2.2 | 29.4 | 92.4 | 71.9 | 71.2 | 92.3 | 78.5 | 2.68 | 2.31 | 2.81 | 2.6 | |
| GLM-4.6Family=ZHIPU2026.04 | 75.9 | 2.71 | 30.6 | 92.5 | 80.4 | 71.9 | 88.7 | 80.3 | 3.45 | 3.26 | 3.54 | 3.42 | |
| GPT-4.1Family=OPENAI2026.04 | 75.2 | 2.45 | 30.6 | 93.6 | 75.9 | 74.6 | 88 | 79.5 | 3.07 | 2.92 | 3.45 | 3.15 | |
| Kimi-K2Family=MOONSHOT2026.04 | 74.8 | 2.79 | 31.8 | 92.5 | 70.2 | 73.3 | 90.4 | 77.9 | 3.67 | 3.33 | 3.38 | 3.46 | |
| Doubao 1.6Family=BYTEDANCE, Level=medium2026.04 | 72.9 | 2.24 | 27.6 | 93.1 | 76.9 | 67.3 | 88.3 | 77.5 | 2.71 | 2.56 | 3.38 | 2.88 | |
| V3.1Family=DEEPSEEK, Thinking Mode=thinking2026.04 | 72.6 | 2.56 | 30.7 | 93.2 | 78.1 | 63.5 | 91.2 | 77.6 | 3.24 | 2.91 | 3.43 | 3.19 | |
| Doubao 1.6Family=BYTEDANCE, Level=low2026.04 | 71.8 | 2.07 | 27.4 | 92.8 | 76.7 | 68 | 88.8 | 77.8 | 2.59 | 2.47 | 3.33 | 2.8 | |
| R1Family=DEEPSEEK2026.04 | 70.8 | 2.44 | 25.9 | 92.5 | 78.3 | 72.5 | 85 | 78.6 | 2.99 | 2.87 | 3.29 | 3.05 | |
| Doubao 1.6Family=BYTEDANCE, Level=high2026.04 | 70.3 | 2.32 | 28.7 | 93 | 68.5 | 69.5 | 88.1 | 75.4 | 2.69 | 2.57 | 3.36 | 2.87 | |
| Gemini 2.5 ProFamily=GOOGLE2026.04 | 70.1 | 2.24 | 28.1 | 92.2 | 71.7 | 64.6 | 80.5 | 72.3 | 2.65 | 2.42 | 3.18 | 2.75 | |
| V3.1Family=DEEPSEEK, Thinking Mode=no-think2026.04 | 69.2 | 2.29 | 31.2 | 93.1 | 72.4 | 65.3 | 87.5 | 75.1 | 3.55 | 3.27 | 3.38 | 3.4 | |
| GPT-5Family=OPENAI2026.04 | 68.3 | 2.26 | 23 | 88.9 | 71.3 | 69.4 | 87.7 | 76.1 | 2.5 | 2.43 | 3.17 | 2.7 | |
| Qwen3-30BFamily=QWEN, Thinking Mode=thinking2026.04 | 65.8 | 2.51 | 25 | 89.5 | 77.9 | 62.4 | 89.9 | 76.7 | 2.65 | 2.42 | 2.76 | 2.42 | |
| Qwen3-235BFamily=QWEN, Thinking Mode=thinking2026.04 | 65.6 | 2.05 | 24.5 | 89.7 | 68.1 | 67.2 | 92.4 | 75.9 | 2.62 | 2.43 | 2.77 | 2.4 | |
| V3-0324Family=DEEPSEEK2026.04 | 59.1 | 1.87 | 27 | 91.7 | 77.5 | 62.7 | 74.5 | 71.5 | 2.84 | 2.66 | 3.13 | 2.87 |