Decode on AMD Ryzen AI 9 HX 370 CPU
225Tokens/s (1K Context)Granite-4.0-350M
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Granite-4.0-350MBatch size=1, Backend=llama.cpp, Quantization=Q4_0, Hardware=AMD Ryzen AI 9 HX 370 CPU2025.11 | 225 | 166.2 | |
| LFM2-350MBatch size=1, Backend=llama.cpp, Quantization=Q4_0, Hardware=AMD Ryzen AI 9 HX 370 CPU2025.11 | 207.7 | 170.5 | |
| Granite-4.0-H-350MBatch size=1, Backend=llama.cpp, Quantization=Q4_0, Hardware=AMD Ryzen AI 9 HX 370 CPU2025.11 | 134.7 | 123.2 | |
| LFM2-700MBatch size=1, Backend=llama.cpp, Quantization=Q4_0, Hardware=AMD Ryzen AI 9 HX 370 CPU2025.11 | 134.6 | 118.4 | |
| Qwen3-0.6BBatch size=1, Backend=llama.cpp, Quantization=Q4_0, Hardware=AMD Ryzen AI 9 HX 370 CPU2025.11 | 116 | 53.4 | |
| Gemma-3-1BBatch size=1, Backend=llama.cpp, Quantization=Q4_0, Hardware=AMD Ryzen AI 9 HX 370 CPU2025.11 | 103.6 | 99.3 | |
| LFM2-1.2BBatch size=1, Backend=llama.cpp, Quantization=Q4_0, Hardware=AMD Ryzen AI 9 HX 370 CPU2025.11 | 99.7 | 89 | |
| Llama-3.2-1BBatch size=1, Backend=llama.cpp, Quantization=Q4_0, Hardware=AMD Ryzen AI 9 HX 370 CPU2025.11 | 97.3 | 73.5 | |
| LFM2-8B-A1BBatch size=1, Backend=llama.cpp, Quantization=Q4_0, Hardware=AMD Ryzen AI 9 HX 370 CPU2025.11 | 74.9 | 69.1 | |
| Granite-4.0-1BBatch size=1, Backend=llama.cpp, Quantization=Q4_0, Hardware=AMD Ryzen AI 9 HX 370 CPU2025.11 | 64.5 | 43.6 | |
| Qwen3-1.7BBatch size=1, Backend=llama.cpp, Quantization=Q4_0, Hardware=AMD Ryzen AI 9 HX 370 CPU2025.11 | 60.8 | 38.5 | |
| Granite-4.0-H-1BBatch size=1, Backend=llama.cpp, Quantization=Q4_0, Hardware=AMD Ryzen AI 9 HX 370 CPU2025.11 | 59.6 | 57.1 | |
| Granite-4.0-H-TinyBatch size=1, Backend=llama.cpp, Quantization=Q4_0, Hardware=AMD Ryzen AI 9 HX 370 CPU2025.11 | 55 | 52.1 | |
| LFM2-2.6BBatch size=1, Backend=llama.cpp, Quantization=Q4_0, Hardware=AMD Ryzen AI 9 HX 370 CPU2025.11 | 50.3 | 46.8 | |
| SmolLM3-3BBatch size=1, Backend=llama.cpp, Quantization=Q4_0, Hardware=AMD Ryzen AI 9 HX 370 CPU2025.11 | 42.4 | 33.4 | |
| Llama-3.2-3BBatch size=1, Backend=llama.cpp, Quantization=Q4_0, Hardware=AMD Ryzen AI 9 HX 370 CPU2025.11 | 38.3 | 28.2 | |
| Granite-4.0-MicroBatch size=1, Backend=llama.cpp, Quantization=Q4_0, Hardware=AMD Ryzen AI 9 HX 370 CPU2025.11 | 36.7 | 28.9 | |
| Granite-4.0-H-MicroBatch size=1, Backend=llama.cpp, Quantization=Q4_0, Hardware=AMD Ryzen AI 9 HX 370 CPU2025.11 | 34.7 | 33.8 | |
| Gemma-3-4BBatch size=1, Backend=llama.cpp, Quantization=Q4_0, Hardware=AMD Ryzen AI 9 HX 370 CPU2025.11 | 33 | 31.6 | |
| Qwen3-4BBatch size=1, Backend=llama.cpp, Quantization=Q4_0, Hardware=AMD Ryzen AI 9 HX 370 CPU2025.11 | 31 | 22.2 |