Diagnostic Troubleshooting on Forum-based Troubleshooting Dataset Mechanical 1.0 (test)
71.58Diagnostic ScoreSIA-all
Evaluation Results
| Method | Links | |
|---|---|---|
| SIA-allBackbone=gemini-3.5, Diagnostic Configuration=SIA-all2026.06 | 71.58 | |
| SIA-allBackbone=gpt-5.5, Diagnostic Configuration=SIA-all2026.06 | 70.02 | |
| SIA-topBackbone=gemini-3.5, Diagnostic Configuration=SIA-top2026.06 | 65.42 | |
| SIA-topBackbone=gpt-5.5, Diagnostic Configuration=SIA-top2026.06 | 64.41 | |
| THKBackbone=gpt-5.5, Diagnostic Configuration=THK2026.06 | 49.52 | |
| THKBackbone=gemini-3.5, Diagnostic Configuration=THK2026.06 | 47.83 | |
| BASBackbone=gpt-5.5, Diagnostic Configuration=BAS2026.06 | 37.8 | |
| BASBackbone=gemini-3.5, Diagnostic Configuration=BAS2026.06 | 36.93 |