Language Modeling Evaluation on Eight benchmark LLM tasks
49,781.23Throughput (Tokens/s)Heterogeneous Digital-AIMC framework
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Heterogeneous Digital-AIMC frameworkParam. in Digital=5.37% (het.gen), Modules in Digital=Dense, Batch size=322026.03 | 49,781.23 | 123.92 | 62.02 | 60.73 | 56.71 | |
| Heterogeneous Digital-AIMC frameworkParam. in Digital=17.01% (het.gen), Modules in Digital=Dense + 12.5% experts, Batch size=322026.03 | 24,924.12 | 62.2 | 62.38 | 61.49 | 58.6 | |
| Heterogeneous Digital-AIMC frameworkParam. in Digital=28.65% (het.gen), Modules in Digital=Dense + 25.0% experts, Batch size=322026.03 | 14,513.52 | 36.25 | 62.5 | 61.82 | 59.6 | |
| Full Digital (FP-16)Param. in Digital=100% (FP-16), Modules in Digital=—, Batch size=322026.03 | 4,220.07 | 10.55 | 63.17 | — | — | |
| Full AnalogParam. in Digital=0% (analog), Modules in Digital=None, Batch size=322026.03 | 768.41 | 23,949.07 | 61.05 | 58.45 | 49.74 |