Fine-grained multimodal reasoning on FGExpertBench
74.88Object AccuracyKFRA
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| KFRACategory=Agent-Based, Base Model=GLM-4.5V-12B2026.03 | 74.88 | 74.49 | 77.55 | 71.33 | 75.33 | 75.29 | 74.81 | |
| Doubao-1.5-Vision-ProCategory=Commercial2026.03 | 71.92 | 66.32 | 73.47 | 67.33 | 71.33 | 68.47 | 69.81 | |
| KFRACategory=Agent-Based, Base Model=Qwen3-VL-8B2026.03 | 69.54 | 72.45 | 83.67 | 65.33 | 70.67 | 69.41 | 71.85 | |
| Gemini-2.5-FlashCategory=Commercial2026.03 | 68.96 | 69.39 | 71.43 | 69.33 | 68.67 | 72.12 | 69.98 | |
| KFRACategory=Agent-Based, Base Model=Qwen2.5-VL-7B2026.03 | 68.47 | 64.29 | 75.51 | 68 | 64.67 | 65.76 | 67.78 | |
| GPT-4oCategory=Commercial2026.03 | 66.5 | 65.31 | 67.35 | 61.33 | 68.46 | 61.19 | 65.03 | |
| GLM-4.5V-12BCategory=Open-Source, Parameters=12B2026.03 | 65.52 | 64.29 | 67.34 | 64.67 | 65.33 | 65.29 | 65.41 | |
| Qwen3-VL-A22BCategory=Open-Source, Parameters=A22B2026.03 | 64.53 | 61.22 | 73.47 | 66.67 | 67.33 | 66.24 | 66.58 | |
| GLM-4.1V-9BCategory=Open-Source, Parameters=9B2026.03 | 62.56 | 63.27 | 57.14 | 64.67 | 59.33 | 58.71 | 60.95 | |
| Qwen-VL-PlusCategory=Commercial2026.03 | 60.1 | 57.14 | 75.51 | 66 | 56 | 61.53 | 62.71 | |
| Qwen2.5-VL-72BCategory=Open-Source, Parameters=72B2026.03 | 57.64 | 51.02 | 69.39 | 69.33 | 66.67 | 63.65 | 62.95 | |
| Qwen3-VL-8BCategory=Open-Source, Parameters=8B2026.03 | 51.72 | 63.27 | 75.51 | 53.33 | 56.67 | 56.71 | 59.54 | |
| GLM-4V-PlusCategory=Commercial2026.03 | 47.29 | 45.92 | 63.27 | 64.67 | 58 | 48.35 | 54.58 | |
| Gemma3-it-4BCategory=Open-Source, Parameters=4B2026.03 | 45.81 | 51.02 | 55.1 | 51.33 | 45.33 | 54 | 50.43 | |
| InternVL3-8BCategory=Open-Source, Parameters=8B2026.03 | 42.36 | 52.04 | 61.22 | 58 | 55.33 | 52.24 | 53.53 | |
| InternVL3.5-8BCategory=Open-Source, Parameters=8B2026.03 | 39.41 | 47.96 | 65.31 | 61.33 | 54.66 | 51.18 | 53.31 | |
| Phi-3.5-4BCategory=Open-Source, Parameters=4B2026.03 | 36.45 | 46.94 | 51.02 | 47.33 | 56 | 45.76 | 47.47 | |
| Qwen2.5-VL-7BCategory=Open-Source, Parameters=7B2026.03 | 33.5 | 42.86 | 63.27 | 46.67 | 34 | 41.53 | 48.64 | |
| VSACategory=Agent-Based2026.03 | 31.68 | 44.9 | 51.02 | 31.33 | 38.67 | 37.81 | 39.24 | |
| LLaVA-Next-8BCategory=Open-Source, Parameters=8B2026.03 | 31.53 | 51.02 | 57.14 | 44 | 36.67 | 42.24 | 43.77 | |
| LLaVA-OneVision-7BCategory=Open-Source, Parameters=7B2026.03 | 31.03 | 36.73 | 48.98 | 40.67 | 34.67 | 44.27 | 39.39 | |
| MMSearchCategory=Agent-Based2026.03 | 27.09 | 39.8 | 38.78 | 26 | 48 | 41.65 | 36.81 |