Long-context Visual Question Answering on Visual HayStack Short Window (test)
98.2Accuracy (1 Image)SFT(LoRA)
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| SFT(LoRA)Model Parameters=3B2025.12 | 98.2 | 91.88 | 87.48 | 68.89 | 31.5 | |
| SFT(LoRA)Model Parameters=7B2025.12 | 97.78 | 92.92 | 85.8 | 84.73 | 35.92 | |
| LAidModel Parameters=3B2025.12 | 96.83 | 83.34 | 74.29 | 63.27 | 20.1 | |
| LAidModel Parameters=7B2025.12 | 92.83 | 83.26 | 80.46 | 74.09 | 24.1 | |
| YaRNModel Parameters=3B2025.12 | 86.27 | 72.86 | 57.79 | 55.74 | 2.8 | |
| BaseModel Parameters=3B2025.12 | 85.91 | 65.7 | 62.09 | 52.16 | — | |
| BaseModel Parameters=32B2025.12 | 83.79 | 79.11 | 74.71 | 73.34 | — | |
| BaseModel Parameters=7B2025.12 | 80.22 | 68.45 | 62.19 | 57.21 | — | |
| YaRNModel Parameters=7B2025.12 | 80.03 | 63.78 | 62.09 | 55.96 | -2.5 | |
| SelfExtendModel Parameters=7B2025.12 | 78.53 | 62.58 | 58.69 | 53.01 | -5.9 | |
| SelfExtendModel Parameters=3B2025.12 | 77.89 | 64.69 | 54.11 | 47.95 | -7.9 |