Language Modeling on Intel Lunar Lake On-device (PPL, FPS, Latency)
11.22Perplexity (PPL)FP16
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| FP16Model=Mamba-1.4B (CPU), Quantization=FP162026.04 | 11.22 | 5.2 | 2.6 | 384 | |
| KL-MPModel=Mamba-1.4B (CPU), Config=p052026.04 | 11.22 | 1.4 | 5.6 | 178 | |
| INT8Model=Mamba-1.4B (CPU), Quantization=INT82026.04 | 11.25 | 1.3 | 5.1 | 196 | |
| FP16Model=Mamba-130M (CPU), Quantization=FP162026.04 | 21.59 | 493 | 22.6 | 44 | |
| KL-MPModel=Mamba-130M (CPU), Config=p052026.04 | 21.61 | 136 | 35.9 | 28 | |
| INT8Model=Mamba-130M (CPU), Quantization=INT82026.04 | 25.32 | 126 | 35.5 | 28 | |
| FP16Model=Mamba2-130M (GPU), Quantization=FP162026.04 | 46.45 | 81 | 0.02 | 60,006 | |
| KL-MPModel=Mamba2-130M (GPU), Config=p022026.04 | 46.46 | 45 | 0.29 | 3,417 | |
| INT8†Model=Mamba2-130M (GPU), Quantization=INT8†2026.04 | 53.03 | 125 | — | — | |
| INT4Model=Mamba-1.4B (CPU), Quantization=INT42026.04 | 60.55 | 723 | 5.3 | 190 | |
| INT4*Model=Mamba-130M (CPU), Quantization=INT4*2026.04 | 300,000,000,000,000,000,000,000,000,000,000,000 | 84 | 39.7 | 25 |