Classification on TorchSight Eval-1000
95Category AccuracyBeam q4_K_M
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Beam q4_K_MType=Local (LoRA), Time=4.4 s, Temperature=02026.05 | 95 | 93.5 | 48.2 | |
| Beam f16Type=Local (LoRA), Time=4.6 s, Temperature=02026.05 | 93.2 | 91.5 | 51.1 | |
| Beam q8_0Type=Local (LoRA), Time=3.2 s, Temperature=02026.05 | 93 | 91.2 | 51.4 | |
| Qwen 3.5 27B baseType=Local (no LoRA), Time=~40 s, Temperature=02026.05 | 86.3 | 84 | 19 | |
| Claude Sonnet 4Type=Commercial API, Time=~6 s, Temperature=02026.05 | 79.9 | 77.3 | 23 | |
| Claude Opus 4Type=Commercial API, Time=~13 s, Temperature=02026.05 | 79.9 | 77.3 | 22.5 | |
| GPT-5Type=Commercial API, Time=~24 s2026.05 | 76.9 | 74.2 | 11.6 | |
| Gemini 2.5 ProType=Commercial API, Time=~14 s, Temperature=02026.05 | 75.4 | 72.6 | 21 | |
| Regex-only (48 pat.)Type=Rule-based, Time=0.2 ms2026.05 | 52.7 | 49.6 | — |