Inference Efficiency on Samsung Galaxy S25 Qualcomm Snapdragon 8 Elite SoC inference v1.0
1,067Prefill Throughput (1K) (tokens/s)LFM2-350M
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| LFM2-350MBatch size=1, Quantization format=Q4.0, Framework=llama.cpp, Compute backend=CPU, Continuation tokens=1002025.11 | 1,067 | 657 | 194.1 | 143.8 | |
| Granite-4.0-H-350MBatch size=1, Quantization format=Q4.0, Framework=llama.cpp, Compute backend=CPU, Continuation tokens=1002025.11 | 784 | 594 | 150.7 | 119.3 | |
| Granite-4.0-350MBatch size=1, Quantization format=Q4.0, Framework=llama.cpp, Compute backend=CPU, Continuation tokens=1002025.11 | 528 | 210 | 132.9 | 70.7 | |
| LFM2-700MBatch size=1, Quantization format=Q4.0, Framework=llama.cpp, Compute backend=CPU, Continuation tokens=1002025.11 | 522 | 341 | 104.2 | 80.2 | |
| Gemma-3-1BBatch size=1, Quantization format=Q4.0, Framework=llama.cpp, Compute backend=CPU, Continuation tokens=1002025.11 | 377 | 295 | 67.5 | 67.1 | |
| LFM2-1.2BBatch size=1, Quantization format=Q4.0, Framework=llama.cpp, Compute backend=CPU, Continuation tokens=1002025.11 | 335 | 222 | 69.8 | 55.5 | |
| Qwen3-0.6BBatch size=1, Quantization format=Q4.0, Framework=llama.cpp, Compute backend=CPU, Continuation tokens=1002025.11 | 318 | 136 | 76.7 | 41.8 | |
| Llama-3.2-1BBatch size=1, Quantization format=Q4.0, Framework=llama.cpp, Compute backend=CPU, Continuation tokens=1002025.11 | 229 | 130 | 54.6 | 37.8 | |
| Granite-4.0-H-1BBatch size=1, Quantization format=Q4.0, Framework=llama.cpp, Compute backend=CPU, Continuation tokens=1002025.11 | 186 | 159 | 46.1 | 44.1 | |
| Granite-4.0-1BBatch size=1, Quantization format=Q4.0, Framework=llama.cpp, Compute backend=CPU, Continuation tokens=1002025.11 | 147 | 79 | 41.3 | 25 | |
| LFM2-2.6BBatch size=1, Quantization format=Q4.0, Framework=llama.cpp, Compute backend=CPU, Continuation tokens=1002025.11 | 143 | 116 | 33.8 | 30 | |
| Qwen3-1.7BBatch size=1, Quantization format=Q4.0, Framework=llama.cpp, Compute backend=CPU, Continuation tokens=1002025.11 | 140 | 98 | 39.7 | 26.9 | |
| SmolLM3-3BBatch size=1, Quantization format=Q4.0, Framework=llama.cpp, Compute backend=CPU, Continuation tokens=1002025.11 | 98 | 66 | 26.1 | 19.2 | |
| Granite-4.0-H-TinyBatch size=1, Quantization format=Q4.0, Framework=llama.cpp, Compute backend=CPU, Continuation tokens=1002025.11 | 86 | 86 | 39.2 | 37.4 | |
| LFM2-8B-A1BBatch size=1, Quantization format=Q4.0, Framework=llama.cpp, Compute backend=CPU, Continuation tokens=1002025.11 | 85 | 76 | 48.6 | 41.9 | |
| Granite-4.0-H-MicroBatch size=1, Quantization format=Q4.0, Framework=llama.cpp, Compute backend=CPU, Continuation tokens=1002025.11 | 83 | 77 | 24.9 | 23.5 | |
| Llama-3.2-3BBatch size=1, Quantization format=Q4.0, Framework=llama.cpp, Compute backend=CPU, Continuation tokens=1002025.11 | 79 | 51 | 24.2 | 15.8 | |
| Gemma-3-4BBatch size=1, Quantization format=Q4.0, Framework=llama.cpp, Compute backend=CPU, Continuation tokens=1002025.11 | 72 | 63 | 18.3 | 17.9 | |
| Granite-4.0-MicroBatch size=1, Quantization format=Q4.0, Framework=llama.cpp, Compute backend=CPU, Continuation tokens=1002025.11 | 65 | 38 | 20.2 | 12.1 | |
| Qwen3-4BBatch size=1, Quantization format=Q4.0, Framework=llama.cpp, Compute backend=CPU, Continuation tokens=1002025.11 | 57 | 35 | 17.2 | 11.4 |