On-device LLM Inference on Knowledge, Code, and Math prompts on iPhone 16 Pro
0.46Prefill TTFT (256 tokens)MobileMoE-S
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| MobileMoE-SMem (GB)=0.68, Backend=ExecuTorch+XNNPACK, Quantization (Weights/Activations)=INT4 weights, INT8 dynamic activations2026.05 | 0.46 | 1.02 | 2.14 | 5.23 | 13.86 | 39.91 | 204.6 | 180.5 | 148.4 | 90.2 | 54.2 | 32.2 | |
| MobileMoE-MMem (GB)=1.48, Backend=ExecuTorch+XNNPACK, Quantization (Weights/Activations)=INT4 weights, INT8 dynamic activations2026.05 | 1.01 | 2.23 | 4.71 | 10.82 | 31.43 | 83.54 | 106.8 | 95.3 | 76.2 | 51 | 27.9 | 17.2 | |
| MobileLLM-ProMem (GB)=0.55, Backend=ExecuTorch+XNNPACK, Quantization (Weights/Activations)=INT4 weights, INT8 dynamic activations2026.05 | 1.29 | 2.9 | 6.03 | 14.13 | 39.08 | 122.42 | 61 | 55.1 | 48.5 | 32.4 | 17.9 | 10.6 | |
| MobileMoE-LMem (GB)=2.75, Backend=ExecuTorch+XNNPACK, Quantization (Weights/Activations)=INT4 weights, INT8 dynamic activations2026.05 | 1.86 | 4.32 | 8.63 | 21.92 | 51.98 | 141.31 | 59.4 | 51.9 | 43.3 | 23.5 | 17.1 | 10.8 |