Prefill Performance on AMD Ryzen AI 9 HX 370 CPU
7,534Prefill Throughput (1K Context) (tokens/s)LFM2-350M
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| LFM2-350MBatch size=1, Backend=llama.cpp, Quantization=Q4_0, Hardware=AMD Ryzen AI 9 HX 370 CPU2025.11 | 7,534 | 5,540 | |
| Granite-4.0-350MBatch size=1, Backend=llama.cpp, Quantization=Q4_0, Hardware=AMD Ryzen AI 9 HX 370 CPU2025.11 | 5,499 | 2,938 | |
| Granite-4.0-H-350MBatch size=1, Backend=llama.cpp, Quantization=Q4_0, Hardware=AMD Ryzen AI 9 HX 370 CPU2025.11 | 4,568 | 4,250 | |
| LFM2-700MBatch size=1, Backend=llama.cpp, Quantization=Q4_0, Hardware=AMD Ryzen AI 9 HX 370 CPU2025.11 | 4,214 | 3,311 | |
| Gemma-3-1BBatch size=1, Backend=llama.cpp, Quantization=Q4_0, Hardware=AMD Ryzen AI 9 HX 370 CPU2025.11 | 3,972 | 3,809 | |
| Qwen3-0.6BBatch size=1, Backend=llama.cpp, Quantization=Q4_0, Hardware=AMD Ryzen AI 9 HX 370 CPU2025.11 | 3,594 | 2,204 | |
| Llama-3.2-1BBatch size=1, Backend=llama.cpp, Quantization=Q4_0, Hardware=AMD Ryzen AI 9 HX 370 CPU2025.11 | 2,912 | 1,890 | |
| LFM2-1.2BBatch size=1, Backend=llama.cpp, Quantization=Q4_0, Hardware=AMD Ryzen AI 9 HX 370 CPU2025.11 | 2,784 | 2,302 | |
| Qwen3-1.7BBatch size=1, Backend=llama.cpp, Quantization=Q4_0, Hardware=AMD Ryzen AI 9 HX 370 CPU2025.11 | 2,019 | 1,491 | |
| Granite-4.0-1BBatch size=1, Backend=llama.cpp, Quantization=Q4_0, Hardware=AMD Ryzen AI 9 HX 370 CPU2025.11 | 1,823 | 1,256 | |
| Granite-4.0-H-1BBatch size=1, Backend=llama.cpp, Quantization=Q4_0, Hardware=AMD Ryzen AI 9 HX 370 CPU2025.11 | 1,565 | 1,523 | |
| LFM2-2.6BBatch size=1, Backend=llama.cpp, Quantization=Q4_0, Hardware=AMD Ryzen AI 9 HX 370 CPU2025.11 | 1,335 | 1,171 | |
| LFM2-8B-A1BBatch size=1, Backend=llama.cpp, Quantization=Q4_0, Hardware=AMD Ryzen AI 9 HX 370 CPU2025.11 | 1,320 | 1,185 | |
| SmolLM3-3BBatch size=1, Backend=llama.cpp, Quantization=Q4_0, Hardware=AMD Ryzen AI 9 HX 370 CPU2025.11 | 1,248 | 982 | |
| Llama-3.2-3BBatch size=1, Backend=llama.cpp, Quantization=Q4_0, Hardware=AMD Ryzen AI 9 HX 370 CPU2025.11 | 1,179 | 916 | |
| Gemma-3-4BBatch size=1, Backend=llama.cpp, Quantization=Q4_0, Hardware=AMD Ryzen AI 9 HX 370 CPU2025.11 | 1,119 | 1,054 | |
| Granite-4.0-H-TinyBatch size=1, Backend=llama.cpp, Quantization=Q4_0, Hardware=AMD Ryzen AI 9 HX 370 CPU2025.11 | 1,050 | 1,046 | |
| Granite-4.0-H-MicroBatch size=1, Backend=llama.cpp, Quantization=Q4_0, Hardware=AMD Ryzen AI 9 HX 370 CPU2025.11 | 932 | 896 | |
| Granite-4.0-MicroBatch size=1, Backend=llama.cpp, Quantization=Q4_0, Hardware=AMD Ryzen AI 9 HX 370 CPU2025.11 | 929 | 608 | |
| Qwen3-4BBatch size=1, Backend=llama.cpp, Quantization=Q4_0, Hardware=AMD Ryzen AI 9 HX 370 CPU2025.11 | 861 | 628 |