Multi-modal Needle-In-A-Haystack on MMNeedle COCO2014-based (test)
42.5Exact Accuracy (8x8)LLaVA-Llama-3-8B + BEFT
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| LLaVA-Llama-3-8B + BEFTM (Number of image panels in haystack)=40, Mapping=each 8 x 8 maps to four 4 x 4, Decoding strategy=BERAG2026.04 | 42.5 | — | — | — | |
| GPT-4oM (Number of image panels in haystack)=102026.04 | 1 | 97 | 81.8 | 26.9 | |
| Claude 3 OpusM (Number of image panels in haystack)=102026.04 | 0 | 66.9 | 4.6 | 0.4 | |
| LLaVA-Llama-3-8BM (Number of image panels in haystack)=102026.04 | 0 | 0 | 0 | 0 | |
| LLaVA-Llama-3-8B + BEFTM (Number of image panels in haystack)=102026.04 | 0 | 97.1 | 86.8 | 41.4 |