Event Sequencing on VECTOR L2 (Ne=8) 1.0 (test)
8,500EM (Exact Match)Human
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Human2025.12 | 8,500 | 91.5 | 94 | 94 | |
| Gemini 1.5 Pro2025.12 | 5,633 | 61.21 | 69.05 | 69.16 | |
| GPT-4o2025.12 | 5,367 | 63.76 | 72.06 | 72.31 | |
| MECoTSize=7B2025.12 | 833 | 19.81 | 35.58 | 40.46 | |
| LV-OVSize=7B2025.12 | 800 | 19.45 | 33.06 | 38.7 | |
| NVILASize=8B2025.12 | 700 | 18.41 | 31.56 | 34.91 | |
| LongVUSize=7B2025.12 | 533 | 16.18 | 27.92 | 31.77 | |
| LongVASize=7B2025.12 | 433 | 9.62 | 19.27 | 22.1 | |
| V-LM2Size=7B2025.12 | 300 | 9.42 | 19.74 | 23.93 | |
| LongLVSize=7B2025.12 | 267 | 7.33 | 14.48 | 16.81 | |
| VILA-1.5Size=8B2025.12 | 167 | 4.41 | 10.79 | 12.33 | |
| Chance2025.12 | 98 | 5.56 | 16.28 | 19.44 | |
| Human2025.12 | 80 | 96.25 | 96.88 | 96.88 | |
| Gemini 1.5 Pro2025.12 | 59.33 | 81.42 | 86.79 | 87.79 | |
| GPT-4o2025.12 | 51.67 | 87 | 91.88 | 92.38 | |
| OryxSize=7B2025.12 | 33 | 1.31 | 4.25 | 4.38 | |
| MECoTSize=7B2025.12 | 4.33 | 47 | 66.79 | 81.21 | |
| NVILASize=8B2025.12 | 0.33 | 28.12 | 57.58 | 74.08 | |
| LV-OVSize=7B2025.12 | 0.33 | 28.88 | 57.88 | 79.58 | |
| Chance2025.12 | 0 | 5 | 23.63 | 40 | |
| LongVASize=7B2025.12 | 0 | 17.04 | 41.96 | 66.08 | |
| OryxSize=7B2025.12 | 0 | 11.12 | 17.88 | 21.12 | |
| VILA-1.5Size=8B2025.12 | 0 | 6.38 | 25.42 | 42.96 | |
| V-LM2Size=7B2025.12 | 0 | 14.62 | 36.96 | 71 | |
| LongLVSize=7B2025.12 | 0 | 6.5 | 24.38 | 42 | |
| LongVUSize=7B2025.12 | 0 | 28.25 | 51.04 | 61.58 |