Video Question Answering on Minerva-Ego 1.0 (test)
91.8MCQ AccuracyHuman performance
Evaluation results
| Method | Links | |
|---|---|---|
| Human performance# Frames=all, Thinking=Yes2026.05 | 91.8 | |
| Gemini 2.5 Pro# Frames=64, Thinking=Yes2026.05 | 40.1 | |
| GPT-5# Frames=64, Thinking=Yes2026.05 | 37.6 | |
| Gemini 2.5 Flash# Frames=64, Thinking=Yes2026.05 | 35.6 | |
| Gemini 2.5 Flash# Frames=64, Thinking=No2026.05 | 31.7 | |
| GPT-4.1# Frames=64, Thinking=No2026.05 | 30.8 | |
| Claude Sonnet 4# Frames=64, Thinking=Yes2026.05 | 30.5 | |
| Qwen-3# Frames=all @1fps, Thinking=Yes2026.05 | 29.3 | |
| Random# Frames=-, Thinking=-2026.05 | 20 |