LLM Inference on Llama 3.2 Samsung Galaxy S25 Ultra 1B (test)
2,813.19Prefill Min Throughput (tokens/sec)ET QNN
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| ET QNNHardware=NPU, GS=322026.05 | 2,813.19 | 2,976.74 | 46.5 | 46.57 | 1,434 | |
| QAIRTHardware=NPU, GS=322026.05 | 2,277.9 | 2,392.34 | 52.03 | 52.72 | 1,229 | |
| ET VulkanHardware=GPU, GS=1282026.05 | 1,207.55 | 1,207.55 | 66.25 | 66.41 | 676 | |
| llama.cppHardware=GPU, GS=322026.05 | 1,064.8 | 1,092.7 | 36.7 | 42 | 728 | |
| ET VulkanHardware=GPU, GS=322026.05 | 927.54 | 930.91 | 59.19 | 59.4 | 920 | |
| ET XNNPACKHardware=CPU, GS=1282026.05 | 649.75 | 658.1 | 71.69 | 71.97 | 742 | |
| ET XNNPACKHardware=CPU, GS=322026.05 | 524.59 | 528.93 | 65.88 | 67.1 | 821 | |
| llama.cppHardware=CPU, GS=322026.05 | 512.7 | 537.8 | 65.6 | 66.5 | 728 | |
| ONNXHardware=CPU, GS=1282026.05 | 490.63 | 538.2 | 74.6 | 76.56 | 659 | |
| llama.cppHardware=NPU, GS=322026.05 | 329.9 | 374.4 | 23.6 | 25.6 | 728 | |
| ONNXHardware=CPU, GS=322026.05 | 284.12 | 328.76 | 60.37 | 64.73 | 769 | |
| LiteRTHardware=CPU, GS=1282026.05 | 185.35 | 190.26 | 34.3 | 34.56 | 612 | |
| LiteRTHardware=CPU, GS=322026.05 | 143.49 | 149.63 | 31.36 | 33.03 | 667 |