Video Retrieval on RVMS-Bench 1.0 (test)
64GHuman
Evaluation Results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| HumanProtocol=Rule-only baseline2026.02 | 64 | — | 28 | 24 | 88 | 24 | 56 | 64 | 60 | 80 | 54.2 | |
| HumanProtocol=Model + Rule (RACLO)2026.02 | 64 | — | — | — | 88 | — | — | 72 | 60 | 80 | 72.8 | |
| Gemini-3-ProProtocol=Model + Rule (RACLO), Availability=Closed Source2026.02 | 45.6 | — | — | — | 58.8 | — | — | 63.1 | 56.9 | 61.2 | 57 | |
| Gemini-2.5-ProProtocol=Model + Rule (RACLO), Availability=Closed Source2026.02 | 36.2 | — | — | — | 28.8 | — | — | 34.4 | 30 | 24.4 | 31 | |
| Qwen3-VL-235BProtocol=Model + Rule (RACLO), Availability=Open Source, Version=Instruct2026.02 | 32.5 | — | — | — | 42.5 | — | — | 40 | 43.8 | 25 | 36.6 | |
| GPT-5-MiniProtocol=Model + Rule (RACLO), Availability=Closed Source2026.02 | 31.2 | — | — | — | 27.5 | — | — | 28.8 | 28.8 | 27.5 | 28.8 | |
| Qwen2.5-VL-72BProtocol=Model + Rule (RACLO), Availability=Open Source, Version=Instruct2026.02 | 29.4 | — | — | — | 23.1 | — | — | 33.8 | 26.2 | 28.8 | 28.1 | |
| Hunyuan-VideoProtocol=Model + Rule (RACLO), Availability=Closed Source2026.02 | 28.8 | — | — | — | 22.5 | — | — | 27.5 | 23.8 | 19.4 | 24.4 | |
| GPT-5.2Protocol=Model + Rule (RACLO), Availability=Closed Source2026.02 | 26.2 | — | — | — | 36.9 | — | — | 38.1 | 31.2 | 36.2 | 33.8 | |
| Claude-4.5-SonnetProtocol=Model + Rule (RACLO), Availability=Closed Source2026.02 | 23.8 | — | — | — | 24.4 | — | — | 26.2 | 21.9 | 23.8 | 24 | |
| Qwen3-VL-235BProtocol=Rule-only baseline, Availability=Open Source, Version=Instruct2026.02 | 21.9 | — | 15.1 | 12.4 | 13.1 | 15 | 16.1 | 11.9 | 23.1 | 30 | 17.6 | |
| Gemini-2.5-ProProtocol=Rule-only baseline, Availability=Closed Source2026.02 | 18.8 | — | 13.1 | 10.6 | 15 | 12.5 | 13.8 | 18.1 | 16.2 | 17.5 | 15.1 | |
| GPT-4oProtocol=Model + Rule (RACLO), Availability=Closed Source2026.02 | 18.8 | — | — | — | 19.4 | — | — | 21.9 | 17.5 | 18.8 | 19.2 | |
| Qwen2.5-VL-72BProtocol=Rule-only baseline, Availability=Open Source, Version=Instruct2026.02 | 16.9 | — | 11.9 | 10 | 11.9 | 11.2 | 12.5 | 17.5 | 15 | 16.2 | 13.7 | |
| GPT-5-MiniProtocol=Rule-only baseline, Availability=Closed Source2026.02 | 16.1 | — | 1.9 | 1.9 | 10.6 | 4.4 | 4 | 7 | 13.8 | 13.1 | 8.1 | |
| Hunyuan-VideoProtocol=Rule-only baseline, Availability=Closed Source2026.02 | 15.6 | — | 10.6 | 8.8 | 12.5 | 10 | 11.2 | 15 | 13.1 | 14.4 | 12.4 | |
| Gemini-3-ProProtocol=Rule-only baseline, Availability=Closed Source2026.02 | 14.4 | — | 23.8 | 16.9 | 16.9 | 20 | 23.8 | 31.9 | 15.6 | 22.5 | 20.6 | |
| GPT-5.2Protocol=Rule-only baseline, Availability=Closed Source2026.02 | 14.4 | — | 3.2 | 1.6 | 12.9 | 6.9 | 5 | 13.1 | 16.2 | 15 | 9.8 | |
| Claude-4.5-SonnetProtocol=Rule-only baseline, Availability=Closed Source2026.02 | 12.5 | — | 10.6 | 8.8 | 12.5 | 10 | 11.2 | 14.4 | 11.2 | 13.1 | 11.6 | |
| GPT-4oProtocol=Rule-only baseline, Availability=Closed Source2026.02 | 10.6 | — | 7.5 | 6.2 | 10 | 7.5 | 8.1 | 11.9 | 10 | 11.2 | 9.2 |