Energy Efficiency Analysis on Neural Network Operations and Components Suite
0.0004Energy (nJ) - LoihiNEXUS
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| NEXUSComponent=Embedding Lookup, Granularity=per output element, Hardware=Loihi 22026.01 | 0.0004 | 64 | 168,000 | 32 | 16 | |
| NEXUSComponent=NOT, Granularity=per bit-operation, Hardware=Loihi 22026.01 | 0.012 | — | — | 1 | 0.5 | |
| NEXUSComponent=AND / OR, Granularity=per bit-operation, Hardware=Loihi 22026.01 | 0.024 | — | — | 1 | 1 | |
| NEXUSComponent=FP32 Adder, Granularity=per FP32 operation, Hardware=Loihi 22026.01 | 0.04 | 1.3 | 33 | 3,348 | 1,674 | |
| NEXUSComponent=FP32 Multiplier, Granularity=per FP32 operation, Hardware=Loihi 22026.01 | 0.048 | 1.3 | 27 | 4,089 | 2,045 | |
| NEXUSComponent=XOR, Granularity=per bit-operation, Hardware=Loihi 22026.01 | 0.059 | — | — | 5 | 2.5 | |
| NEXUSComponent=MUX, Granularity=per bit-operation, Hardware=Loihi 22026.01 | 0.059 | — | — | 5 | 2.5 | |
| NEXUSComponent=Sin / Cos, Granularity=per element, Hardware=Loihi 22026.01 | 0.068 | 12.8 | 187 | 5,800 | 2,900 | |
| NEXUSComponent=Exp, Granularity=per element, Hardware=Loihi 22026.01 | 0.073 | 12.8 | 175 | 6,200 | 3,100 | |
| NEXUSComponent=Sigmoid, Granularity=per element, Hardware=Loihi 22026.01 | 0.081 | 12.8 | 158 | 6,850 | 3,425 | |
| NEXUSComponent=Tanh, Granularity=per element, Hardware=Loihi 22026.01 | 0.084 | 12.8 | 153 | 7,100 | 3,550 | |
| NEXUSComponent=RMSNorm, dim=256, Granularity=per element, Hardware=Loihi 22026.01 | 0.1 | 89 | 890 | 8,500 | 4,250 | |
| NEXUSComponent=FP32 Square Root, Granularity=per FP32 operation, Hardware=Loihi 22026.01 | 0.105 | 6.4 | 61 | 8,920 | 4,460 | |
| NEXUSComponent=FP32 Reciprocal, Granularity=per FP32 operation, Hardware=Loihi 22026.01 | 0.12 | 6.4 | 53 | 10,200 | 5,100 | |
| NEXUSComponent=SiLU, Granularity=per element, Hardware=Loihi 22026.01 | 0.129 | 12.8 | 99 | 10,939 | 5,470 | |
| NEXUSComponent=SiLU Activation (FFN), Granularity=per token, Hardware=Loihi 22026.01 | 0.132 | 13 | 99 | 11,200 | 5,600 | |
| NEXUSComponent=LayerNorm, dim=256, Granularity=per element, Hardware=Loihi 22026.01 | 0.146 | 128 | 877 | 12,400 | 6,200 | |
| NEXUSComponent=FP32 Divider, Granularity=per FP32 operation, Hardware=Loihi 22026.01 | 0.147 | 6.4 | 44 | 12,450 | 6,225 | |
| NEXUSComponent=Full Adder, Granularity=per bit-operation, Hardware=Loihi 22026.01 | 0.153 | — | — | 13 | 6.5 | |
| NEXUSComponent=GELU, Granularity=per element, Hardware=Loihi 22026.01 | 0.179 | 12.8 | 71 | 15,200 | 7,600 | |
| NEXUSComponent=Linear (64x64), Granularity=per output element, Hardware=Loihi 22026.01 | 3.09 | 160 | 52 | 262,000 | 131,000 | |
| NEXUSComponent=RoPE, Granularity=per token, Hardware=Loihi 22026.01 | 22.4 | 1,280 | 57 | 1,900,000 | 950,000 | |
| NEXUSComponent=Linear (256x256), Granularity=per output element, Hardware=Loihi 22026.01 | 49.6 | 2,560 | 52 | 4,200,000 | 2,100,000 | |
| NEXUSComponent=Attention Scores, d=256, heads=4, Granularity=per token, Hardware=Loihi 22026.01 | 49.6 | 2,560 | 52 | 4,200,000 | 2,100,000 | |
| NEXUSComponent=Output Projection, Granularity=per token, Hardware=Loihi 22026.01 | 49.6 | 2,560 | 52 | 4,200,000 | 2,100,000 | |
| NEXUSComponent=Softmax, seq=256, Granularity=per token, Hardware=Loihi 22026.01 | 56.6 | 7,500 | 133 | 4,800,000 | 2,400,000 | |
| NEXUSComponent=QKV Projection, d=256, heads=4, Granularity=per token, Hardware=Loihi 22026.01 | 149 | 7,680 | 52 | 12,600,000 | 6,300,000 | |
| NEXUSComponent=Linear (512x512), Granularity=per output element, Hardware=Loihi 22026.01 | 198 | 10,200 | 52 | 16,800,000 | 8,400,000 | |
| NEXUSComponent=Up Projection, d=256, d_ff=1024, Granularity=per token, Hardware=Loihi 22026.01 | 198 | 10,200 | 52 | 16,800,000 | 8,400,000 | |
| NEXUSComponent=Down Projection, Granularity=per token, Hardware=Loihi 22026.01 | 198 | 10,200 | 52 | 16,800,000 | 8,400,000 | |
| NEXUSComponent=Full Attention, Granularity=per token, Hardware=Loihi 22026.01 | 328 | 21,600 | 66 | 27,700,000 | 13,900,000 | |
| NEXUSComponent=Full FFN, Granularity=per token, Hardware=Loihi 22026.01 | 396 | 20,400 | 52 | 33,600,000 | 16,800,000 | |
| NEXUSComponent=Transformer Block, Granularity=per token, Hardware=Loihi 22026.01 | 724 | 42,000 | 58 | 61,300,000 | 30,700,000 |