Question Answering on Streaming-Eval QA 1.0 (test)
54.2Backward ScoreStreamingHarness-8B
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| StreamingHarness-8BStreaming Harness Adaptation=true, Model Scale=8B2026.06 | 54.2 | 21 | 48.9 | 32.5 | 61.9 | 56.3 | 45.8 | |
| GPT 5.4 (High)Streaming Harness Adaptation=true, Model Scale=High, Step size (Ts)=102026.06 | 26 | 17.4 | 15.1 | 9.9 | 20.2 | 21 | 18.3 | |
| Gemini 3.1 Pro (High)Streaming Harness Adaptation=true, Model Scale=High, Step size (Ts)=102026.06 | 17 | 8.2 | 16.9 | 11.9 | 14.7 | 14.2 | 13.8 | |
| Qwen3-VL-32BStreaming Harness Adaptation=true, Model Scale=32B2026.06 | 17 | 8 | 15.2 | 6.8 | 10.4 | 13.8 | 11.9 | |
| Qwen3-VL-8BStreaming Harness Adaptation=true, Model Scale=8B2026.06 | 16.2 | 5.7 | 13.7 | 7 | 10.2 | 12 | 10.8 | |
| Doubao Seed 2.0 ProStreaming Harness Adaptation=true, Step size (Ts)=102026.06 | 14.6 | 9.7 | 17.9 | 13.2 | 13.1 | 11.3 | 13.3 | |
| Claude Opus 4.6 ThinkingStreaming Harness Adaptation=true, Thinking/Reasoning Mode=true, Step size (Ts)=102026.06 | 12.5 | 6.1 | 10.3 | 8.6 | 11.1 | 8.4 | 9.5 | |
| Qwen3.5-VL-9B (Thinking)Streaming Harness Adaptation=true, Thinking/Reasoning Mode=true, Model Scale=9B2026.06 | 0 | 0 | 0.1 | 0 | 0.2 | 0.3 | 0.1 |