Question Answering on OpenROAD (TTFT ms)
271.87TTFT (ms)H2MT
Evaluation results
| Method | Links | |
|---|---|---|
| H2MTBackbone=SmolLM-135M2026.05 | 271.87 | |
| H2MTBackbone=Mistral-7B2026.05 | 295.17 | |
| H2MTBackbone=Qwen2.5-7B2026.05 | 308.98 | |
| H2MTBackbone=Qwen3-4B2026.05 | 321.18 | |
| H2MTBackbone=Llama3.1-8B2026.05 | 348.56 | |
| H2MTBackbone=Qwen2.5-14B2026.05 | 522.13 | |
| HMTBackbone=SmolLM-135M2026.05 | 566.93 | |
| HMTBackbone=Llama3.1-8B2026.05 | 1,512.08 | |
| HMTBackbone=Qwen2.5-14B2026.05 | 2,250.71 | |
| HMTBackbone=Qwen3-4B2026.05 | 16,888.34 | |
| HMTBackbone=Qwen2.5-7B2026.05 | 18,588.63 | |
| HMTBackbone=Mistral-7B2026.05 | 26,196.64 |