Diagnostic Troubleshooting on Forum-based Troubleshooting Dataset Electrical 1.0 (test)
69.12Diagnostic ScoreSIA-all
Evaluation Results
| Method | Links | |
|---|---|---|
| SIA-allBackbone=gemini-3.5, Diagnostic Configuration=SIA-all2026.06 | 69.12 | |
| SIA-allBackbone=gpt-5.5, Diagnostic Configuration=SIA-all2026.06 | 66.13 | |
| SIA-topBackbone=gemini-3.5, Diagnostic Configuration=SIA-top2026.06 | 63.46 | |
| SIA-topBackbone=gpt-5.5, Diagnostic Configuration=SIA-top2026.06 | 59.02 | |
| THKBackbone=gpt-5.5, Diagnostic Configuration=THK2026.06 | 38.53 | |
| THKBackbone=gemini-3.5, Diagnostic Configuration=THK2026.06 | 36.5 | |
| BASBackbone=gpt-5.5, Diagnostic Configuration=BAS2026.06 | 31.12 | |
| BASBackbone=gemini-3.5, Diagnostic Configuration=BAS2026.06 | 28.48 |