Motion Direction Understanding on MODIRECT-REALBENCH
100SSv2 Top-1 AccHuman
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Human2026.05 | 100 | 99.9 | 99.9 | 99.9 | 100 | |
| LLaVA-Video-7B w/ DeltaDirectmode=LoRA post-tuning2026.05 | 81.5 | 74.8 | 38.8 | 65 | 76.7 | |
| LLaVA-Video-7B w/ DeltaDirectFine-tuning protocol=Instruction tuning, Supervision method=DeltaDirect, Backbone=7B2026.05 | 81.5 | 74.8 | 38.8 | 65 | 76.7 | |
| Full-FT-0.5B w/ DeltaDirectmode=Full fine-tuning, backbone=Qwen2-0.5B + SigLIP2026.05 | 77.8 | 69.4 | 21.7 | 56.3 | 76.9 | |
| Qwen2-0.5B Full-FT w/ DeltaDirectFine-tuning protocol=Full fine-tuning, Supervision method=DeltaDirect, Backbone=Qwen2-0.5B2026.05 | 77.8 | 69.4 | 21.7 | 56.3 | 76.9 | |
| LLaVA-Video-7B w/ MODIRECT-INSTmode=LoRA post-tuning2026.05 | 72.4 | 66.6 | 35.2 | 58.1 | 69.9 | |
| LLaVA-Video-7B w/ MODIRECT-INSTFine-tuning protocol=Instruction tuning, Supervision method=MODIRECT-INST, Backbone=7B2026.05 | 72.4 | 66.6 | 35.2 | 58.1 | 69.9 | |
| InternVL-2.5-4B2026.05 | 66.1 | 50.7 | 26.1 | 47.6 | 38.3 | |
| InternVL-2.5-4BBackbone=4B2026.05 | 66.1 | 50.7 | 26.1 | 47.6 | 38.3 | |
| Qwen3-VL-4B2026.05 | 60.9 | 62.1 | 35 | 52.7 | 51 | |
| Qwen3-VL-4BBackbone=4B2026.05 | 60.9 | 62.1 | 35 | 52.7 | 51 | |
| Full-FT-0.5B w/ VideoChat2 + MODIRECT-INSTmode=Full fine-tuning, backbone=Qwen2-0.5B + SigLIP2026.05 | 59 | 55.2 | 20.6 | 44.9 | 63.7 | |
| Qwen2-0.5B Full-FT w/ MODIRECT-INSTFine-tuning protocol=Full fine-tuning, Supervision method=MODIRECT-INST, Backbone=Qwen2-0.5B2026.05 | 59 | 55.2 | 20.6 | 44.9 | 63.7 | |
| VideoLLaMA3-7B2026.05 | 56.5 | 52.2 | 19.9 | 42.9 | 47.1 | |
| VideoLLaMA3-7BBackbone=7B2026.05 | 56.5 | 52.2 | 19.9 | 42.9 | 47.1 | |
| Full-FT-0.5B w/ VideoChat2mode=Full fine-tuning, backbone=Qwen2-0.5B + SigLIP2026.05 | 53.4 | 54.2 | 19.6 | 42.4 | 32.2 | |
| Qwen2.5-VL-7B2026.05 | 52.6 | 44.2 | 25.3 | 40.7 | 37.2 | |
| Qwen2.5-VL-7BBackbone=7B2026.05 | 52.6 | 44.2 | 25.3 | 40.7 | 37.2 | |
| LLaVA-OneVision-SI-7B2026.05 | 52.5 | 47.5 | 21.6 | 40.5 | 31.7 | |
| LLaVA-Video-7B w/ FlashVID2026.05 | 52.5 | 50.3 | 21.8 | 41.5 | 32 | |
| LLaVA-Video-7B w/ FlashVIDSupervision method=FlashVID, Backbone=7B2026.05 | 52.5 | 50.3 | 21.8 | 41.5 | 32 | |
| LLaVA-OneVision-7B2026.05 | 52.2 | 50.3 | 26.8 | 43.1 | 34.3 | |
| LLaVA-Video-7B2026.05 | 52.2 | 50.3 | 26.8 | 43.1 | 33.3 | |
| LLaVA-OneVision-7BBackbone=7B2026.05 | 52.2 | 50.3 | 26.8 | 43.1 | 34.3 | |
| LLaVA-Video-7BBackbone=7B2026.05 | 52.2 | 50.3 | 26.8 | 43.1 | 33.3 | |
| LLaMA-VID-7B2026.05 | 51.4 | 54.2 | 16.9 | 40.8 | 31.9 | |
| LLaMA-VID-7BBackbone=7B2026.05 | 51.4 | 54.2 | 16.9 | 40.8 | 31.9 | |
| GPT-4o2026.05 | 51 | 52.3 | 34.5 | 45.9 | 44.4 | |
| GPT-4o2026.05 | 51 | 52.3 | 34.5 | 45.9 | 44.4 | |
| LLaVA-NeXT-Video-7B2026.05 | 50.8 | 52.4 | 21.1 | 41.4 | 32.2 | |
| LLaVA-NeXT-Video-7BBackbone=7B2026.05 | 50.8 | 52.4 | 21.1 | 41.4 | 32.2 | |
| Random Chance2026.05 | 50 | 50 | 20 | 40 | 31.4 | |
| VideoChat2-HD-7B2026.05 | 50 | 54.2 | 20.1 | 41.4 | 31.8 | |
| Random Chance2026.05 | 50 | 50 | 20 | 40 | 31.4 | |
| VideoChat2-HD-7BBackbone=7B2026.05 | 50 | 54.2 | 20.1 | 41.4 | 31.8 | |
| Video-LLaVA-7B2026.05 | 49.2 | 49.5 | 14.6 | 37.8 | 31.7 | |
| Video-LLaVA-7BBackbone=7B2026.05 | 49.2 | 49.5 | 14.6 | 37.8 | 31.7 | |
| GPT-4o-mini2026.05 | 48.6 | 51.2 | 22.1 | 40.6 | 33.5 | |
| VideoLLaMA3-2B2026.05 | 46.1 | 49.4 | 21.8 | 39.1 | 33.2 | |
| mPLUG-Owl3-7B2026.05 | 43.4 | 50.3 | 21.3 | 38.3 | 30.9 | |
| Gemini 2.5 Flash2026.05 | 20 | 69.4 | 25.6 | 38.3 | 47 | |
| Gemini 2.5 Flash2026.05 | 20 | 69.4 | 25.6 | 38.3 | 47 |