LLM Inference Efficiency on HumanEval
5,233.11TTFT (ms)ReMoE
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| ReMoEHardware=Jetson Orin NX 16GB, Backend=llama.cpp, Number of parallel requests (-np)=1, Sequence length (-n)=128, Memory mapping (--mmap)=true2026.05 | 5,233.11 | 337.61 | 1.99 | |
| BaselineHardware=Jetson Orin NX 16GB, Backend=llama.cpp, Number of parallel requests (-np)=1, Sequence length (-n)=128, Memory mapping (--mmap)=true2026.05 | 7,185.78 | 672.68 | — |