Error Identification on TRAIL GAIA (dev)
85.8PrecisionTC Judge
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| TC JudgeJudge Type=TC2025.10 | 85.8 | 98.5 | 91.7 | 95.6 | 92 | |
| Tool CallingAbbreviation=TC2025.10 | 85.7 | 46.5 | 60.3 | 51.2 | 72.7 | |
| EE JudgeJudge Type=EE2025.10 | 78.7 | 92.1 | 84.9 | 89.1 | 84.1 | |
| Execution EfficiencyAbbreviation=EE2025.10 | 75.2 | 71.4 | 73.3 | 72.2 | 74.7 | |
| Tool SelectionAbbreviation=TS2025.10 | 75 | 48.4 | 58.8 | 52.1 | 78.2 | |
| TS JudgeJudge Type=TS2025.10 | 73.6 | 98.9 | 84.4 | 92.6 | 88.2 | |
| PQ JudgeJudge Type=PQ2025.10 | 68.2 | 88.2 | 76.9 | 83.3 | 96.9 | |
| Logic ConsistencyAbbreviation=LC2025.10 | 67 | 64.2 | 65.5 | 64.7 | 72 | |
| Plan QualityAbbreviation=PQ2025.10 | 64.7 | 64.7 | 64.7 | 64.7 | 95.9 | |
| LC JudgeJudge Type=LC2025.10 | 63.6 | 80 | 70.9 | 76.1 | 72.7 | |
| Plan AdherenceAbbreviation=PA2025.10 | 63.2 | 78.7 | 70.1 | 75 | 85.8 | |
| PA JudgeJudge Type=PA2025.10 | 54.9 | 91.8 | 68.7 | 80.9 | 82.4 |