Inference Latency on MS MARCO c (test)
0.57Latency p50 (s)TEI RadixMLP
Evaluation results
| Method | Links | ||
|---|---|---|---|
| TEI RadixMLPModel=Qwen3-0.6B2026.01 | 0.57 | 0.63 | |
| TEIModel=Qwen3-0.6B2026.01 | 0.7 | 0.77 | |
| vLLMModel=Qwen3-0.6B, version=v0.13.02026.01 | 0.71 | 0.83 | |
| vLLMModel=Qwen3-4B, version=v0.13.02026.01 | 2.44 | 2.53 | |
| TEI RadixMLPModel=Qwen3-4B2026.01 | 2.52 | 2.78 | |
| TEIModel=Qwen3-4B2026.01 | 3.33 | 3.66 | |
| vLLMModel=Qwen3-8B, version=v0.13.02026.01 | 3.77 | 3.91 | |
| TEI RadixMLPModel=Qwen3-8B2026.01 | 3.88 | 4.28 | |
| TEIModel=Qwen3-8B2026.01 | 5.23 | 5.75 |