Audio-Visual Question Answering on Video Holmes 32 frames
64.4SRStreamOV
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| StreamOVAudio=true2026.05 | 64.4 | 56.2 | 52.4 | 44.5 | 48.5 | 53.6 | 50.4 | 53.1 | |
| Qwen3-Omni-30BAudio=true2026.05 | 60.6 | 62.3 | 48.4 | 35.5 | 50.9 | 51 | 55.2 | 52.8 | |
| Gemini 2.0 FlashAudio=true2026.05 | 56.5 | 54.2 | 43.4 | 44.5 | 43.9 | 55.1 | 50.1 | 49.5 | |
| GPT-4oAudio=false2026.05 | 50 | 49.6 | 38.8 | 30 | 44 | 39.2 | 37 | 42 | |
| Qwen3-VL-30BAudio=false2026.05 | 46.6 | 38.4 | 35.5 | 39 | 39.8 | 35.1 | 35.6 | 38.8 | |
| Gemini 2.5 FlashAudio=true2026.05 | 43.4 | 46.9 | 43.1 | 51 | 37.9 | 43.6 | 39.3 | 43.1 | |
| Qwen2.5-Omni-7BAudio=true2026.05 | 41.8 | 35.1 | 23.4 | 25 | 28.9 | 17.5 | 25.9 | 29 | |
| Qwen2.5-VL-7BAudio=false2026.05 | 34.6 | 21.4 | 15.4 | 12.5 | 17.2 | 12.4 | 15.2 | 19 |