Frame-level attribute tagging on Gold Set N=108
71.7PrecisionGemma-3
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Gemma-3Method Type=Single Scout, Quantization=4-bit (Q4_K_M)2025.12 | 71.7 | 72.9 | 72.3 | 1.787 | 8 | |
| Qwen3-VL + YOLOMethod Type=Single Scout, Object Detection Integration=YOLO, Quantization=4-bit (Q4_K_M)2025.12 | 71.4 | 93.2 | 80.9 | 1.13 | 31.5 | |
| Semantic-Drive (Consensus)Method Type=Neuro-Symbolic Consensus Ensemble, Scouts=Heterogeneous (Qwen3, Kimi-VL, Gemma-3), Consensus Judge=Ministral-3-14B-Instruct-2512, Quantization=4-bit (Q4_K_M)2025.12 | 71.2 | 96.6 | 82 | 0.676 | 60 | |
| Pure VLMBackbone=Qwen3, Object Detection Integration=No YOLO, Status=Ablation 12025.12 | 69.1 | 81.4 | 74.7 | 1.389 | 31 | |
| CLIPBackbone=ViT-L/14, Status=Baseline 2, Mode=Zero-Shot2025.12 | 68.3 | 47.5 | 56 | — | 0.2 | |
| Kimi-VLMethod Type=Single Scout, Quantization=4-bit (Q4_K_M)2025.12 | 47.9 | 28.8 | 36 | 3.28 | 11.1 | |
| Metadata Keyword SearchMethod Type=Metadata Search, Status=Baseline 12025.12 | 40.6 | 60.2 | 48.5 | — | 0 |