Trajectory step classification on Trajectory nine-category label set
95Label AccuracyClaude Haiku 4.5
Evaluation Results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Claude Haiku 4.5Tier=frontier, Mode=aware2026.05 | 95 | 94.7 | 9.2 | 87.9 | 20 | 96.2 | 87.9 | 85.4 | |
| Gemma-4-26B-A4B-itTier=open, Mode=aware2026.05 | 94.3 | 93.7 | 8.9 | 93.4 | 19.8 | 95 | 76.5 | 74.3 | |
| Gemini 3.1 Flash LiteTier=frontier, Mode=aware2026.05 | 92.7 | 92.5 | 9.2 | 91.1 | 19.7 | 93.7 | 85.6 | 82.4 | |
| Ministral-3-14BTier=open, Mode=aware2026.05 | 90.5 | 90.2 | 8.9 | 76.9 | 19 | 91.5 | 86.8 | 80.2 | |
| Qwen3.6-35B-A3BTier=open, Mode=aware2026.05 | 89.5 | 89.7 | 14.6 | 92.6 | 23.3 | 89 | 87 | 80.5 | |
| Claude Sonnet 4.6Tier=frontier, Mode=aware2026.05 | 88.5 | 88.6 | 9.3 | 94.2 | 19.6 | 90.3 | 86.5 | 79.6 | |
| gpt-5.4-miniTier=frontier, Mode=aware2026.05 | 81.8 | 82.7 | 14.6 | 89.1 | 22.3 | 83.8 | 87.3 | 76.2 | |
| gpt-oss-20BTier=open, Mode=aware2026.05 | 68.6 | 66.7 | 10.5 | 85.8 | 18.5 | 78.9 | 81.5 | 66 | |
| gpt-5.4-miniTier=frontier, Mode=blind2026.05 | 61.5 | 61.5 | 10.6 | 82 | 18.5 | — | — | — | |
| Claude Sonnet 4.6Tier=frontier, Mode=blind2026.05 | 61.3 | 61.3 | 9.5 | 90.9 | 18.4 | — | — | — | |
| Claude Haiku 4.5Tier=frontier, Mode=blind2026.05 | 56.9 | 58.6 | 8.4 | 85.9 | 16.9 | — | — | — | |
| Gemini 3.1 Flash LiteTier=frontier, Mode=blind2026.05 | 56.4 | 58.5 | 9.2 | 91.6 | 17.6 | — | — | — | |
| Qwen3.6-35B-A3BTier=open, Mode=blind2026.05 | 55.5 | 56.9 | 9.1 | 82.9 | 17 | — | — | — | |
| Gemma-4-26B-A4B-itTier=open, Mode=blind2026.05 | 54.2 | 56 | 8.6 | 88.3 | 16.8 | — | — | — | |
| gpt-oss-20BTier=open, Mode=blind2026.05 | 53.8 | 52 | 7.8 | 87.8 | 16.1 | — | — | — | |
| Ministral-3-14BTier=open, Mode=blind2026.05 | 53.7 | 56.1 | 8.9 | 77.7 | 16.5 | — | — | — |