3D spatio-temporal QA on Stream3D-Bench
58.8Average AccuracyStream3D-VLM-8B
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Stream3D-VLM-8BTTFT Latency=62ms, End2End Latency=0.39s, Memory Usage=36.6G, Image Resolution=504×3922026.06 | 58.8 | 59.8 | 67.9 | 50.5 | 61.4 | 61.5 | 54.3 | 58.1 | 66.7 | 49.4 | 86.7 | |
| Stream3D-VLM-4BTTFT Latency=43ms, End2End Latency=0.24s, Memory Usage=20.7G, Image Resolution=504×3922026.06 | 54.6 | 58.1 | 61.2 | 45.2 | 58.8 | 57.3 | 52.8 | 55.3 | 60.6 | 41.9 | 75.4 | |
| Qwen2.5-VL-7BFine-tuned=true, TTFT Latency=106ms, End2End Latency=0.53s, Memory Usage=36.8G, Image Resolution=504×3922026.06 | 47.8 | 45.6 | 56.6 | 45.4 | 45.7 | 44.6 | 48.7 | 46.6 | 57.4 | 39.8 | 73.1 | |
| GPT-5Image Resolution=1296×968, Model Category=Proprietary Models (API)2026.06 | 35 | 18.7 | 44.1 | 33.6 | 38 | 46.2 | 52.8 | 12.9 | 30.6 | 37.7 | 61.7 | |
| VideoLLM-online-8BFine-tuned=true, TTFT Latency=120ms, End2End Latency=1.29s, Memory Usage=28.4G, Image Resolution=384×3842026.06 | 34.6 | 28.8 | 40.9 | 27.8 | 31.3 | 37.9 | 44.6 | 38.2 | 32.7 | 29.2 | 70.2 | |
| Qwen2.5-VL-32BTTFT Latency=560ms, End2End Latency=5.87s, Memory Usage=91.6G, Image Resolution=1008×7842026.06 | 29.3 | 14.1 | 40.3 | 35.7 | 28.7 | 37.6 | 43.2 | 22.4 | 30.2 | 11.5 | 35.2 | |
| GPT-4oImage Resolution=1296×968, Model Category=Proprietary Models (API)2026.06 | 28 | 10.5 | 36.3 | 24.9 | 31.1 | 34.8 | 51.1 | 10.2 | 29.9 | 23 | 55.9 | |
| InternVL3.5-8BTTFT Latency=1771ms, End2End Latency=6.72s, Memory Usage=43.1G, Image Resolution=1024×7652026.06 | 27 | 21.4 | 36.2 | 17.6 | 28.2 | 38.2 | 42.4 | 25.2 | 25.8 | 8.2 | 23 | |
| InternVL3-8BTTFT Latency=1034ms, End2End Latency=5.65s, Memory Usage=40.5G, Image Resolution=1024×7652026.06 | 24.1 | 22.1 | 34.4 | 6.8 | 30.7 | 31.2 | 40 | 18.3 | 27.2 | 5.9 | 19.8 | |
| Qwen3-VL-32BTTFT Latency=282ms, End2End Latency=3.22s, Memory Usage=65.5G, Image Resolution=1024×7682026.06 | 23.1 | 4.9 | 34.2 | 35.9 | 19.4 | 32 | 31.8 | 7.3 | 29.2 | 13.5 | 66.4 | |
| Qwen2.5-VL-7BTTFT Latency=325ms, End2End Latency=3.16s, Memory Usage=40.0G, Image Resolution=1008×7842026.06 | 22.5 | 11.6 | 36.3 | 13.6 | 23.1 | 33.2 | 39.4 | 11.3 | 28.4 | 5.5 | 19.2 | |
| LLaVA-Video-7BTTFT Latency=650ms, End2End Latency=2.84s, Memory Usage=25.8G, Image Resolution=1152×7682026.06 | 20.4 | 12.7 | 34.6 | 11.4 | 21.6 | 28.6 | 38 | 10.2 | 20.3 | 6.5 | 55.5 | |
| Qwen3-VL-8BTTFT Latency=150ms, End2End Latency=1.56s, Memory Usage=28.8G, Image Resolution=1024×7682026.06 | 19 | 2.5 | 33.2 | 20.2 | 16 | 33.4 | 28.4 | 4.6 | 24.7 | 8.3 | 61.7 |