Diagnostic Troubleshooting on Forum-based Troubleshooting Dataset Hydraulic 1.0 (test)
73.94Diagnostic ScoreSIA-all
Evaluation Results
| Method | Links | |
|---|---|---|
| SIA-allBackbone=gpt-5.5, Diagnostic Configuration=SIA-all2026.06 | 73.94 | |
| SIA-allBackbone=gemini-3.5, Diagnostic Configuration=SIA-all2026.06 | 73.06 | |
| SIA-topBackbone=gpt-5.5, Diagnostic Configuration=SIA-top2026.06 | 68.43 | |
| SIA-topBackbone=gemini-3.5, Diagnostic Configuration=SIA-top2026.06 | 68.1 | |
| THKBackbone=gpt-5.5, Diagnostic Configuration=THK2026.06 | 44.01 | |
| THKBackbone=gemini-3.5, Diagnostic Configuration=THK2026.06 | 42.17 | |
| BASBackbone=gpt-5.5, Diagnostic Configuration=BAS2026.06 | 35.63 | |
| BASBackbone=gemini-3.5, Diagnostic Configuration=BAS2026.06 | 33.79 |