Video Understanding on Video-MME v1.0 (test)
72.4Score (Short)TS-LLaVA
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| TS-LLaVAModel=Qwen2-VL-7B, Ratio=2^-3, Tokens=4320, Frames=2562025.10 | 72.4 | 59.6 | 50.8 | 60.93 | |
| Full (No Compression)Model=Qwen2-VL-7B, Ratio=100%, Tokens=34560, Frames=2562025.10 | 72.1 | 63.2 | 53.9 | 63.07 | |
| FLoCModel=Qwen2-VL-7B, Ratio=2^-3, Tokens=4320, Frames=2562025.10 | 72 | 60.22 | 50.44 | 60.89 | |
| DivPruneModel=Qwen2-VL-7B, Ratio=2^-3, Tokens=4320, Frames=2562025.10 | 71.22 | 59 | 51.78 | 60.67 | |
| SCModel=Qwen2-VL-7B, Ratio=2^-3, Tokens=4320, Frames=2562025.10 | 71.22 | 61 | 51 | 61.07 | |
| LongVUModel=Qwen2-VL-7B, Ratio=2^-3, Tokens=4320, Frames=2562025.10 | 71.11 | 57.67 | 47.89 | 58.89 | |
| Frame UniformModel=Qwen2-VL-7B, Ratio=2^-3, Tokens=4320, Frames=322025.10 | 71 | 56 | 48.89 | 58.63 | |
| DivPruneModel=Qwen2-VL-7B, Ratio=2^-4, Tokens=2160, Frames=2562025.10 | 70.67 | 57 | 50.33 | 59.3 | |
| K-medoidsModel=Qwen2-VL-7B, Ratio=2^-3, Tokens=4320, Frames=2562025.10 | 70.33 | 59.78 | 50.89 | 60.33 | |
| TS-LLaVAModel=Qwen2-VL-7B, Ratio=2^-4, Tokens=2160, Frames=2562025.10 | 70 | 55.7 | 48.4 | 58.04 | |
| K-medoidsModel=Qwen2-VL-7B, Ratio=2^-4, Tokens=2160, Frames=2562025.10 | 69.67 | 56.89 | 50.78 | 59.11 | |
| FLoCModel=Qwen2-VL-7B, Ratio=2^-5, Tokens=1080, Frames=2562025.10 | 69.33 | 55.56 | 48.22 | 57.7 | |
| FLoCModel=Qwen2-VL-7B, Ratio=2^-4, Tokens=2160, Frames=2562025.10 | 69.22 | 58 | 51 | 59.41 | |
| K-meansModel=Qwen2-VL-7B, Ratio=2^-3, Tokens=4320, Frames=2562025.10 | 69 | 55 | 46.78 | 56.93 | |
| SCModel=Qwen2-VL-7B, Ratio=2^-4, Tokens=2160, Frames=2562025.10 | 68.78 | 57.89 | 51 | 59.22 | |
| DivPruneModel=Qwen2-VL-7B, Ratio=2^-5, Tokens=1080, Frames=2562025.10 | 68.22 | 53.67 | 49.33 | 57.07 | |
| SCModel=Qwen2-VL-7B, Ratio=2^-5, Tokens=1080, Frames=2562025.10 | 67.89 | 53.56 | 48.33 | 56.59 | |
| TS-LLaVAModel=Qwen2-VL-7B, Ratio=2^-5, Tokens=1080, Frames=2562025.10 | 67.4 | 54.3 | 48.3 | 56.7 | |
| Frame UniformModel=Qwen2-VL-7B, Ratio=2^-4, Tokens=2160, Frames=162025.10 | 67.22 | 53 | 47.22 | 55.81 | |
| TS-LLaVAModel=Qwen2-VL-2B, Ratio=2^-3, Tokens=4320, Frames=2562025.10 | 66.89 | 52.33 | 45.33 | 54.85 | |
| SCModel=Qwen2-VL-2B, Ratio=2^-3, Tokens=4320, Frames=2562025.10 | 66.44 | 52.44 | 47.33 | 55.41 | |
| FLoCModel=Qwen2-VL-2B, Ratio=2^-3, Tokens=4320, Frames=2562025.10 | 66.11 | 52.44 | 47 | 55.19 | |
| LongVUModel=Qwen2-VL-7B, Ratio=2^-4, Tokens=2160, Frames=2562025.10 | 65.89 | 54 | 47.78 | 55.89 | |
| K-meansModel=Qwen2-VL-7B, Ratio=2^-4, Tokens=2160, Frames=2562025.10 | 65.78 | 52.89 | 47.22 | 55.3 | |
| DivPruneModel=Qwen2-VL-2B, Ratio=2^-3, Tokens=4320, Frames=2562025.10 | 65.44 | 50.78 | 45.44 | 53.89 | |
| SCModel=Qwen2-VL-2B, Ratio=2^-4, Tokens=2160, Frames=2562025.10 | 65.22 | 51.67 | 44.78 | 53.89 | |
| K-medoidsModel=Qwen2-VL-7B, Ratio=2^-5, Tokens=1080, Frames=2562025.10 | 65.22 | 51.44 | 46.33 | 54.33 | |
| Frame UniformModel=Qwen2-VL-2B, Ratio=2^-3, Tokens=4320, Frames=322025.10 | 65.11 | 49.33 | 43.67 | 52.7 | |
| LongVUModel=Qwen2-VL-2B, Ratio=2^-3, Tokens=4320, Frames=2562025.10 | 65 | 52.44 | 47.11 | 54.85 | |
| Full (No Compression)Model=Qwen2-VL-2B, Ratio=100%, Tokens=34560, Frames=2562025.10 | 64.89 | 50.56 | 45.89 | 53.78 | |
| FLoCModel=Qwen2-VL-2B, Ratio=2^-4, Tokens=2160, Frames=2562025.10 | 64.67 | 52.78 | 45.67 | 54.37 | |
| TS-LLaVAModel=Qwen2-VL-2B, Ratio=2^-4, Tokens=2160, Frames=2562025.10 | 64.44 | 50.56 | 43.56 | 52.85 | |
| DivPruneModel=Qwen2-VL-2B, Ratio=2^-4, Tokens=2160, Frames=2562025.10 | 64.44 | 48.67 | 44.44 | 52.52 | |
| FLoCModel=Qwen2-VL-2B, Ratio=2^-5, Tokens=1080, Frames=2562025.10 | 64.22 | 49 | 45 | 52.74 | |
| LongVUModel=Qwen2-VL-7B, Ratio=2^-5, Tokens=1080, Frames=2562025.10 | 63.7 | 50.2 | 48.7 | 54.19 | |
| K-meansModel=Qwen2-VL-2B, Ratio=2^-3, Tokens=4320, Frames=2562025.10 | 63.67 | 49.67 | 44.11 | 52.48 | |
| K-medoidsModel=Qwen2-VL-2B, Ratio=2^-3, Tokens=4320, Frames=2562025.10 | 63.44 | 51.56 | 44.89 | 53.3 | |
| PoolingModel=Qwen2-VL-7B, Ratio=2^-3, Tokens=4320, Frames=2562025.10 | 63.33 | 50.89 | 46 | 53.41 | |
| SCModel=Qwen2-VL-2B, Ratio=2^-5, Tokens=1080, Frames=2562025.10 | 63 | 49.56 | 45 | 52.52 | |
| TS-LLaVAModel=Qwen2-VL-2B, Ratio=2^-5, Tokens=1080, Frames=2562025.10 | 62.78 | 47.33 | 43.67 | 51.26 | |
| Frame UniformModel=Qwen2-VL-7B, Ratio=2^-5, Tokens=1080, Frames=82025.10 | 62.78 | 49.89 | 46.89 | 53.19 | |
| K-medoidsModel=Qwen2-VL-2B, Ratio=2^-4, Tokens=2160, Frames=2562025.10 | 62.67 | 49 | 41.56 | 51.07 | |
| Frame UniformModel=Qwen2-VL-2B, Ratio=2^-4, Tokens=2160, Frames=162025.10 | 62.44 | 47.22 | 42.44 | 50.7 | |
| DivPruneModel=Qwen2-VL-2B, Ratio=2^-5, Tokens=1080, Frames=2562025.10 | 61.78 | 47 | 43.89 | 50.89 | |
| LongVUModel=Qwen2-VL-2B, Ratio=2^-4, Tokens=2160, Frames=2562025.10 | 61.56 | 47.56 | 43.78 | 50.96 | |
| K-meansModel=Qwen2-VL-2B, Ratio=2^-4, Tokens=2160, Frames=2562025.10 | 61.56 | 47.11 | 42.11 | 50.26 | |
| K-meansModel=Qwen2-VL-7B, Ratio=2^-5, Tokens=1080, Frames=2562025.10 | 61.11 | 50.56 | 46.56 | 52.74 | |
| K-medoidsModel=Qwen2-VL-2B, Ratio=2^-5, Tokens=1080, Frames=2562025.10 | 58.89 | 45.56 | 41.11 | 48.52 | |
| Frame UniformModel=Qwen2-VL-2B, Ratio=2^-5, Tokens=1080, Frames=82025.10 | 58.11 | 44.67 | 41.56 | 48.11 | |
| LongVUModel=Qwen2-VL-2B, Ratio=2^-5, Tokens=1080, Frames=2562025.10 | 57.78 | 43.67 | 41.89 | 47.78 | |
| PoolingModel=Qwen2-VL-7B, Ratio=2^-4, Tokens=2160, Frames=2562025.10 | 57.78 | 48.33 | 43.78 | 49.96 | |
| K-meansModel=Qwen2-VL-2B, Ratio=2^-5, Tokens=1080, Frames=2562025.10 | 56.33 | 44.33 | 40.44 | 47.04 | |
| PoolingModel=Qwen2-VL-2B, Ratio=2^-3, Tokens=4320, Frames=2562025.10 | 55.78 | 44.33 | 42 | 47.37 | |
| PoolingModel=Qwen2-VL-7B, Ratio=2^-5, Tokens=1080, Frames=2562025.10 | 53.67 | 47.22 | 42.56 | 47.81 | |
| PoolingModel=Qwen2-VL-2B, Ratio=2^-4, Tokens=2160, Frames=2562025.10 | 47.56 | 39.67 | 39.22 | 42.15 | |
| PoolingModel=Qwen2-VL-2B, Ratio=2^-5, Tokens=1080, Frames=2562025.10 | 44.44 | 38.89 | 38.56 | 40.63 |