Streaming Narration on Streaming-Eval
70Win Rate vs. Claude Opus 4.6 ThinkingGemini 3.1 Pro (High)
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Gemini 3.1 Pro (High)Streaming harness framework adaptation=true, Video frame rate=1 FPS, Ts=10 s, M=4, L=8, Source=Closed-source2026.06 | 70 | 62 | 35.5 | 69 | 59.1 | |
| StreamingHarness-8BStreaming harness framework adaptation=true, Video frame rate=1 FPS, Ts=200 s, M=5, L=42, Source=Open-source2026.06 | 65 | 65 | 44 | 71.5 | 61.4 | |
| GPT 5.4 (High)Streaming harness framework adaptation=true, Video frame rate=1 FPS, Ts=10 s, M=4, L=8, Source=Closed-source2026.06 | 52.5 | 12.5 | 18 | 44 | 31.8 | |
| Doubao Seed 2.0 ProStreaming harness framework adaptation=true, Video frame rate=1 FPS, Ts=10 s, M=4, L=8, Source=Closed-source2026.06 | 19.5 | 11.5 | 7 | 10 | 12 | |
| Claude Opus 4.6 ThinkingStreaming harness framework adaptation=true, Video frame rate=1 FPS, Ts=10 s, M=4, L=8, Source=Closed-source2026.06 | 14.5 | 10.5 | 10 | 26 | 15.3 | |
| Qwen3-VL-8BStreaming harness framework adaptation=true, Video frame rate=1 FPS, Ts=200 s, M=5, L=42, Source=Open-source2026.06 | 4 | 4 | 2.5 | 4 | 3.6 | |
| Qwen3-VL-32BStreaming harness framework adaptation=true, Video frame rate=1 FPS, Ts=200 s, M=5, L=42, Source=Open-source2026.06 | 3 | 3 | 0.5 | 2 | 2.1 | |
| Qwen3.5-9B (Thinking)Streaming harness framework adaptation=true, Video frame rate=1 FPS, Ts=200 s, M=5, L=42, Source=Open-source2026.06 | 0.5 | 0.5 | 0 | 1 | 0.5 |