Spectral Visual Inspection on SpecVI-Bench in-distribution 1.0 (test)
93.1CV AccuracySpec-o3-7B
Evaluation Results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Spec-o3-7BBase Model=Qwen2.5-VL-7B2026.01 | 93.1 | 81 | 92.5 | 80.2 | 94.2 | 84.5 | 90.6 | 83.4 | 73.2 | 53.6 | 88.7 | 76.5 | |
| CarbonNetCategory=Specialist Deep Learning Model2026.01 | 92.7 | 76.6 | 95.2 | 87.5 | 85.7 | 64.4 | 83.1 | 52 | 61.8 | 40.8 | 83.7 | 64.3 | |
| AstroCLIPCategory=Specialist Deep Learning Model2026.01 | 92.4 | 74.7 | 94 | 84.1 | 88.7 | 62.4 | 80.5 | 52.3 | 80.9 | 48.8 | 87.3 | 64.5 | |
| Spec-o3-3BBase Model=Qwen2.5-VL-3B2026.01 | 92 | 80.7 | 88 | 75.9 | 92.6 | 84 | 93 | 82.7 | 78.5 | 43.2 | 88.8 | 73.3 | |
| GaiaNetCategory=Specialist Deep Learning Model2026.01 | 91.1 | 67.2 | 95.4 | 87.1 | 89.4 | 70.3 | 85.5 | 51.8 | 83.8 | 48.2 | 89 | 64.9 | |
| o3Category=Proprietary VLM2026.01 | 88.8 | 57.1 | 87.1 | 53.1 | 88.1 | 53.3 | 81.7 | 60 | 81.9 | 37.8 | 85.5 | 52.3 | |
| GPT-4.1Category=Proprietary VLM2026.01 | 57.9 | 27.7 | 51.9 | 31.7 | 60.2 | 36.1 | 20.6 | 29.2 | 52.3 | 24.4 | 48.6 | 29.8 | |
| Qwen3-VL-8BCategory=Open-source VLM, Variant=Instruct2026.01 | 39.7 | 33.6 | 29.8 | 27 | 29.4 | 28.4 | 22.9 | 27.3 | 52.9 | 19.7 | 34.9 | 27.2 | |
| Qwen2.5-VL-3BCategory=Open-source VLM, Variant=Instruct2026.01 | 30.6 | 25.8 | 25.3 | 20 | 19 | 26.9 | 17.8 | 28.7 | 48.2 | 28.6 | 28.2 | 26.1 | |
| Qwen2.5-VL-7BCategory=Open-source VLM, Variant=Instruct2026.01 | 28.4 | 25.4 | 59.8 | 31.5 | 19.4 | 27.3 | 19.7 | 29 | 25.8 | 28.1 | 30.6 | 28.3 | |
| Qwen3-VL-30B-A3BCategory=Open-source VLM, Variant=Instruct2026.01 | 22.9 | 31.3 | 19.8 | 29.5 | 17.5 | 26.3 | 35.1 | 27.4 | 65.3 | 34.4 | 32.1 | 29.8 |