Video Question Answering on MA-EgoQA
86.44SI ScoreOracle
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| OracleBackbone=Gemini-2.5-Flash, Input context=-2026.03 | 86.44 | 97.6 | 70.21 | 88.15 | 81.34 | 83.8 | |
| OracleBackbone=Qwen3VL-8B-Instruct, Input context=-2026.03 | 76.86 | 80.99 | 57.02 | 77.35 | 69.92 | 73.98 | |
| BM25Input context=8.1k, Mode=RAG Baseline2026.03 | 44.68 | 37.6 | 30.21 | 33.45 | 30.64 | 36.01 | |
| EgoMASBackbone=Qwen3VL-8B-Instruct, Input context=7.4k2026.03 | 43.09 | 39.05 | 24.26 | 35.54 | 40.67 | 37.68 | |
| EgoMASBackbone=Gemini-2.5-Flash, Input context=4.6k2026.03 | 41.49 | 41.32 | 33.62 | 39.37 | 48.19 | 41.41 | |
| Gemini-2.5-FlashInput context=1M, Mode=All Caption Concat2026.03 | 41.22 | 36.36 | 24.26 | 46.59 | 33.98 | 36.93 | |
| EgoMASBackbone=Qwen3VL-8B-Thinking, Input context=5.4k2026.03 | 38.03 | 39.88 | 28.94 | 47.39 | 44.85 | 40.26 | |
| EgoMASBackbone=Qwen2.5VL-7B-Instruct, Input context=5.4k2026.03 | 37.77 | 38.64 | 25.96 | 34.15 | 36.49 | 35.55 | |
| GPT-5Input context=272k, Mode=All Caption Concat2026.03 | 36.17 | 33.88 | 22.55 | 39.71 | 38.72 | 34.81 | |
| DPRInput context=7.8k, Mode=RAG Baseline2026.03 | 31.12 | 28.31 | 18.72 | 26.48 | 22.84 | 26.19 | |
| Qwen3-30b-a3b-instructInput context=240k, Mode=All Caption Concat2026.03 | 29.52 | 27.69 | 16.6 | 23 | 26.46 | 25.56 | |
| WorldMM-8BInput context=4.1k, Mode=RAG Baseline2026.03 | 29.26 | 34.5 | 21.7 | 25.09 | 22.56 | 27.63 | |
| gpt-oss-120bInput context=128k, Mode=All Caption Concat2026.03 | 28.72 | 27.48 | 22.55 | 28.22 | 25.63 | 26.82 | |
| Qwen2.5-7B-Instruct-1MInput context=500k, Mode=All Caption Concat2026.03 | 27.13 | 28.93 | 20.85 | 24.04 | 26.18 | 26.08 | |
| VideoRAGInput context=8.0k, Mode=RAG Baseline2026.03 | 27.13 | 26.03 | 20.85 | 25.09 | 25.63 | 25.33 | |
| Qwen2.5-VL-7BInput context=128k, Mode=All Frame Concat2026.03 | 26.33 | 26.45 | 21.28 | 22.22 | 25.07 | 25.22 | |
| VideoChat-FlashInput context=32k, Mode=All Frame Concat2026.03 | 23.67 | 25 | 20 | 24.39 | 20.73 | 23.06 | |
| EgoRAGInput context=-, Mode=RAG Baseline2026.03 | 22.34 | 21.07 | 18.3 | 15.33 | 18.87 | 19.57 | |
| Ego-R1-Agent-3BInput context=-, Mode=RAG Baseline2026.03 | 22.34 | 25.62 | 18.3 | 26.83 | 28.41 | 24.7 | |
| Llama-3.1-Nemotron-8BInput context=1M, Mode=All Caption Concat2026.03 | 20.48 | 22.93 | 20.43 | 22.65 | 21.17 | 21.65 | |
| VideoXL-2Input context=32k, Mode=All Frame Concat2026.03 | 20.21 | 19.83 | 19.15 | 20.91 | 21.73 | 20.39 | |
| RandomInput context=-2026.03 | 20 | 20 | 20 | 20 | 20 | 20 | |
| gpt-oss-20bInput context=128k, Mode=All Caption Concat2026.03 | 19.15 | 28.1 | 23.83 | 26.83 | 25.35 | 24.81 |