Question Answering on Choice QA MCTest
92.38AccuracyBatch (Full)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Batch (Full)Backbone=Qwen-3-4B, Processing Mode=Offline2026.05 | 92.38 | 204.98 | 100 | |
| Batch (Full)Backbone=Qwen2.5-3B-Instruct, Processing Mode=Offline2026.05 | 88.33 | 204.87 | 100 | |
| Proactive-EntrBackbone=Qwen2.5-3B-Instruct, Processing Mode=Streaming2026.05 | 85.83 | 160.46 | 78 | |
| Proactive-EntrBackbone=Qwen-3-4B, Processing Mode=Streaming2026.05 | 83.33 | 154.81 | 75 | |
| Proactive-AttnBackbone=Qwen2.5-3B-Instruct, Processing Mode=Streaming2026.05 | 83.15 | 151.62 | 74 | |
| Proactive-AttnBackbone=Qwen-3-4B, Processing Mode=Streaming2026.05 | 82.15 | 158.56 | 75 | |
| Wait-9Backbone=Qwen-3-4B, Processing Mode=Streaming, Wait-k interval=92026.05 | 47.98 | 9 | 4 | |
| Wait-7Backbone=Qwen2.5-3B-Instruct, Processing Mode=Streaming, Wait-k interval=72026.05 | 47.5 | 7 | 3 | |
| Wait-5Backbone=Qwen-3-4B, Processing Mode=Streaming, Wait-k interval=52026.05 | 47.5 | 5 | 2 | |
| Wait-9Backbone=Qwen2.5-3B-Instruct, Processing Mode=Streaming, Wait-k interval=92026.05 | 47.14 | 9 | 4 | |
| Wait-5Backbone=Qwen2.5-3B-Instruct, Processing Mode=Streaming, Wait-k interval=52026.05 | 45.71 | 5 | 2 | |
| Wait-7Backbone=Qwen-3-4B, Processing Mode=Streaming, Wait-k interval=72026.05 | 45.36 | 7 | 3 | |
| Wait-3Backbone=Qwen2.5-3B-Instruct, Processing Mode=Streaming, Wait-k interval=32026.05 | 43.1 | 3 | 1 | |
| Wait-3Backbone=Qwen-3-4B, Processing Mode=Streaming, Wait-k interval=32026.05 | 31.79 | 3 | 1 |