Question Answering on QASPER (TTFT (ms))
233.2TTFT (ms)H2MT
Evaluation results
| Method | Links | |
|---|---|---|
| H2MTBackbone=Qwen2.5-7B2026.05 | 233.2 | |
| H2MTBackbone=Llama3.1-8B2026.05 | 267.93 | |
| H2MTBackbone=SmolLM-135M2026.05 | 289 | |
| H2MTBackbone=Qwen3-4B2026.05 | 306.99 | |
| H2MTBackbone=Mistral-7B2026.05 | 328.62 | |
| H2MTBackbone=Qwen2.5-14B2026.05 | 364.88 | |
| HMTBackbone=SmolLM-135M2026.05 | 666.5 | |
| HMTBackbone=Qwen3-4B2026.05 | 1,696.1 | |
| HMTBackbone=Mistral-7B2026.05 | 2,202.6 | |
| HMTBackbone=Llama3.1-8B2026.05 | 2,400.3 | |
| HMTBackbone=Qwen2.5-7B2026.05 | 3,056.91 | |
| HMTBackbone=Qwen2.5-14B2026.05 | 3,841.9 |