Question Answering on NarrativeQA (TTFT)
355.12TTFT (ms)H2MT
Evaluation results
| Method | Links | |
|---|---|---|
| H2MTBackbone=Qwen3-4B2026.05 | 355.12 | |
| H2MTBackbone=Qwen2.5-7B2026.05 | 361.38 | |
| H2MTBackbone=Llama3.1-8B2026.05 | 364.12 | |
| H2MTBackbone=Qwen2.5-14B2026.05 | 643.57 | |
| H2MTBackbone=Mistral-7B2026.05 | 718.3 | |
| HMTBackbone=SmolLM-135M2026.05 | 816.82 | |
| H2MTBackbone=SmolLM-135M2026.05 | 1,127.05 | |
| HMTBackbone=Qwen3-4B2026.05 | 2,405.94 | |
| HMTBackbone=Qwen2.5-7B2026.05 | 3,056.91 | |
| HMTBackbone=Mistral-7B2026.05 | 3,078.94 | |
| HMTBackbone=Qwen2.5-14B2026.05 | 3,537.69 | |
| HMTBackbone=Llama3.1-8B2026.05 | 3,714.65 |