CPU Inference Performance Evaluation on Qwen3-30B-A3B
16.2Memory Usage (GB)Llama.cpp (CPU)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Llama.cpp (CPU)Bit Width=A8W4, Hardware=Intel(R) Xeon(R) w7-3445 CPU, Threads=202026.04 | 16.2 | 20.1 | |
| CodeQuant (CPU)Bit Width=A8W4, Hardware=Intel(R) Xeon(R) w7-3445 CPU, Threads=202026.04 | 16.5 | 15.9 | |
| Llama.cpp (CPU)Bit Width=BF16, Hardware=Intel(R) Xeon(R) w7-3445 CPU, Threads=202026.04 | 56.9 | 66.1 |