Inference Efficiency on On-device Samsung Galaxy S25 (1k, 2k, 4k context)
0.84Prefill TTFT (1k)LFM2 350M
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| LFM2 350MParameter Count=350M, Hardware=Samsung Galaxy S25, CPU Threads=4, Quantization=4-bit weight, 8-bit activation2026.03 | 0.84 | 2.18 | 4.52 | 147.54 | 96.9 | 63.6 | |
| MobileLLM-Flash 350MParameter Count=350M, Hardware=Samsung Galaxy S25, CPU Threads=4, Quantization=4-bit weight, 8-bit activation2026.03 | 0.91 | 2.78 | 5.33 | 165.56 | 112.58 | 95.55 | |
| Gemma3 270MParameter Count=270M, Hardware=Samsung Galaxy S25, CPU Threads=4, Quantization=4-bit weight, 8-bit activation2026.03 | 0.99 | 3.13 | 5.64 | 123.71 | 82.85 | 52.4 | |
| MobileLLM-Flash 650MParameter Count=650M, Hardware=Samsung Galaxy S25, CPU Threads=4, Quantization=4-bit weight, 8-bit activation2026.03 | 1.62 | 3.34 | 8.48 | 96.64 | 85.35 | 60.74 | |
| LFM2 700MParameter Count=700M, Hardware=Samsung Galaxy S25, CPU Threads=4, Quantization=4-bit weight, 8-bit activation2026.03 | 2.05 | 6.01 | 8.24 | 86.3 | 53.57 | 44.36 | |
| MobileLLM-Flash 1.4BParameter Count=1.4B, Hardware=Samsung Galaxy S25, CPU Threads=4, Quantization=4-bit weight, 8-bit activation2026.03 | 3.4 | 9.08 | 16.5 | 60.52 | 42.65 | 34.01 | |
| LFM2 1.2BParameter Count=1.2B, Hardware=Samsung Galaxy S25, CPU Threads=4, Quantization=4-bit weight, 8-bit activation2026.03 | 3.46 | 8.41 | 16.88 | 61.14 | 42.15 | 29.14 | |
| Gemma3 1BParameter Count=1B, Hardware=Samsung Galaxy S25, CPU Threads=4, Quantization=4-bit weight, 8-bit activation2026.03 | 3.55 | 9.29 | 18.86 | 58.58 | 43.11 | 36.29 | |
| Llama3.2 1BParameter Count=1B, Hardware=Samsung Galaxy S25, CPU Threads=4, Quantization=4-bit weight, 8-bit activation2026.03 | 4.51 | 15.09 | 26.95 | 39.01 | 18.13 | 14.34 | |
| Qwen3 0.6BParameter Count=0.6B, Hardware=Samsung Galaxy S25, CPU Threads=4, Quantization=4-bit weight, 8-bit activation2026.03 | 4.63 | 11.59 | 25.4 | 44.56 | 28.67 | 18.48 |