Multi-modal Question Answering on DailyOmni
61.9AccuracyFull Tokens
Evaluation Results
| Method | Links | |
|---|---|---|
| Full TokensBackbone=Qwen2.5-Omni-7B, Target (%)=100, GPU Mem. (GB)=19, Prefill Spd.=1.00x, E2E Spd.=1.00x2026.07 | 61.9 | |
| OmniFocusBackbone=Qwen2.5-Omni-7B, Target (%)=35, GPU Mem. (GB)=18, Prefill Spd.=1.35x, E2E Spd.=1.29x2026.07 | 60.32 | |
| Full TokensBackbone=Qwen2.5-Omni-3B, Target (%)=100, GPU Mem. (GB)=11, Prefill Spd.=1.00x, E2E Spd.=1.00x2026.07 | 60.15 | |
| OmniFocusBackbone=Qwen2.5-Omni-7B, Target (%)=25, GPU Mem. (GB)=18, Prefill Spd.=1.38x, E2E Spd.=1.32x2026.07 | 59.4 | |
| OmniZipBackbone=Qwen2.5-Omni-7B, Target (%)=35, GPU Mem. (GB)=18, Prefill Spd.=1.31x, E2E Spd.=1.24x2026.07 | 58.98 | |
| OmniFocusBackbone=Qwen2.5-Omni-3B, Target (%)=35, GPU Mem. (GB)=10, Prefill Spd.=1.19x, E2E Spd.=1.17x2026.07 | 57.23 | |
| OmniZipBackbone=Qwen2.5-Omni-7B, Target (%)=25, GPU Mem. (GB)=18, Prefill Spd.=1.34x, E2E Spd.=1.27x2026.07 | 56.98 | |
| OmniFocusBackbone=Qwen2.5-Omni-3B, Target (%)=25, GPU Mem. (GB)=10, Prefill Spd.=1.20x, E2E Spd.=1.19x2026.07 | 56.14 | |
| OmniZipBackbone=Qwen2.5-Omni-3B, Target (%)=35, GPU Mem. (GB)=10, Prefill Spd.=1.15x, E2E Spd.=1.15x2026.07 | 55.56 | |
| OmniZipBackbone=Qwen2.5-Omni-3B, Target (%)=25, GPU Mem. (GB)=10, Prefill Spd.=1.09x, E2E Spd.=1.08x2026.07 | 54.72 |