Needle In A Haystack (NIAH) on VNBench 4-try circular evaluation
66.7Overall NIAH ScoreGemini 1.5 Pro
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Gemini 1.5 ProSize (LLM)=-, #Visual Tokens=-2026.03 | 66.7 | 65.1 | |
| QViC-MFSize (LLM)=7B (Qwen2), #Visual Tokens=16, Sampling Rate=2 fps2026.03 | 64.7 | 58.7 | |
| GPT-4oSize (LLM)=-, #Visual Tokens=-2026.03 | 64.4 | 56.3 | |
| QViC-MFSize (LLM)=7B (Qwen2), #Visual Tokens=16, Sampling Rate=1 fps2026.03 | 63 | 56.9 | |
| LLaVA-VideoSize (LLM)=7B (Qwen2), #Visual Tokens=1692026.03 | 62.5 | 40.4 | |
| Video-XLSize (LLM)=7B (Qwen2), #Visual Tokens=162026.03 | 61.6 | — | |
| LLaMA-VIDSize (LLM)=7B (Vicuna-v1.5), #Visual Tokens=22026.03 | 10.8 | 6.4 |