Inference Efficiency on openPangu Embedded Efficiency Benchmark
528Prefill Latency (ms)openPangu-Embedded
Evaluation results
| Method | Links | ||
|---|---|---|---|
| openPangu-EmbeddedPrecision=INT8, Batch size=22025.12 | 528 | 10.55 | |
| openPangu-EmbeddedPrecision=FP16, Batch size=22025.12 | 632 | 16.84 | |
| openPangu-EmbeddedPrecision=INT8, Batch size=42025.12 | 723 | 12.44 | |
| openPangu-EmbeddedPrecision=FP16, Batch size=42025.12 | 921 | 18.75 | |
| openPangu-EmbeddedPrecision=INT8, Batch size=82025.12 | 1,129 | 16.24 | |
| openPangu-EmbeddedPrecision=FP16, Batch size=82025.12 | 1,589 | 22.55 | |
| openPangu-EmbeddedPrecision=INT8, Batch size=162025.12 | 1,975 | 23.83 | |
| openPangu-EmbeddedPrecision=FP16, Batch size=162025.12 | 2,852 | 30.13 | |
| openPangu-EmbeddedPrecision=INT8, Batch size=322025.12 | 3,611 | 39.01 | |
| openPangu-EmbeddedPrecision=FP16, Batch size=322025.12 | 5,419 | 45.31 |