Instruction Following on OmniCap-IF (test)
83.29Overall CSRHuman
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Human2026.06 | 83.29 | 35.31 | 94.83 | 84.19 | 78.23 | 40.19 | 78.38 | 80.05 | 72.43 | |
| Gemini-3.1-Pro2026.06 | 80.65 | 25.82 | 90.45 | 78.65 | 75.02 | 32.45 | 74.15 | 77.45 | 73.4 | |
| Gemini-3-Flash2026.06 | 79.5 | 23.55 | 88.57 | 74.15 | 74.29 | 31.17 | 73.6 | 76.63 | 72.35 | |
| MiMo-V2.52026.06 | 76.22 | 20.5 | 86.4 | 71.81 | 70.37 | 26.75 | 69.82 | 74.73 | 67.68 | |
| MiMo-V2-Omni2026.06 | 74.4 | 17.21 | 80.6 | 62.04 | 70.84 | 26.43 | 70.14 | 73.51 | 68.95 | |
| Qwen3-Omni-30B-A3BMode=Thinking2026.06 | 71.91 | 14.27 | 84.29 | 67.34 | 64.79 | 19.9 | 65.63 | 69.08 | 61.58 | |
| OmniCaptioner-IFParameters=7B2026.06 | 70.73 | 11.46 | 90.39 | 77.92 | 59.43 | 13.59 | 58.71 | 64.71 | 55.62 | |
| OmniCaptioner-IFParameters=3B2026.06 | 66.67 | 7.86 | 87.73 | 73.12 | 54.57 | 9.79 | 55.91 | 60.39 | 50.06 | |
| MiniCPM-o-4.5Parameters=9B2026.06 | 64.69 | 9.27 | 78.6 | 56.04 | 56.7 | 13.07 | 59.24 | 62.64 | 51.86 | |
| Qwen3-Omni-30B-A3BMode=Instruct2026.06 | 62.65 | 7.24 | 77.37 | 54.64 | 54.19 | 10.83 | 58.13 | 59.92 | 49.31 | |
| Qwen2.5-OmniParameters=7B2026.06 | 49.19 | 2.34 | 62.97 | 34.17 | 41.27 | 4.53 | 47.68 | 47.51 | 34.88 | |
| MiniCPM-o-2.6Parameters=8B2026.06 | 47.38 | 1.88 | 62.31 | 31.46 | 38.81 | 3.75 | 46.78 | 44.45 | 32.28 | |
| Qwen2.5-OmniParameters=3B2026.06 | 40.13 | 0.78 | 52.49 | 22.97 | 33.02 | 2.14 | 41.55 | 38.16 | 26.62 | |
| HumanOmniV2Parameters=7B2026.06 | 32.95 | 0.6 | 32.32 | 11.04 | 33.31 | 3.19 | 42.34 | 36.38 | 28.3 | |
| video-SALMONN-2Parameters=7B2026.06 | 32.8 | 0.42 | 41.09 | 13.8 | 28.03 | 1.25 | 34.27 | 33.74 | 22.1 | |
| ARC-Hunyuan-VideoParameters=7B2026.06 | 29.74 | 0.31 | 20.27 | 5.75 | 34.71 | 4.17 | 44.51 | 37.24 | 26.62 | |
| ASID-CaptionerParameters=7B2026.06 | 24.52 | 0.47 | 17.5 | 4.43 | 28.56 | 2.76 | 39.49 | 32.71 | 23.64 |