Training Throughput on GPT2-1.5B
4.1ThroughputDirectQ
Evaluation Results
| Method | Links | |
|---|---|---|
| DirectQNetwork Bandwidth=10 Gbps, Forward quantization (fw)=4-bit, Backward quantization (bw)=8-bit2022.06 | 4.1 | |
| DirectQNetwork Bandwidth=10 Gbps, Forward quantization (fw)=3-bit, Backward quantization (bw)=6-bit2022.06 | 4 | |
| DirectQNetwork Bandwidth=1 Gbps, Forward quantization (fw)=3-bit, Backward quantization (bw)=6-bit2022.06 | 4 | |
| DirectQNetwork Bandwidth=1 Gbps, Forward quantization (fw)=4-bit, Backward quantization (bw)=8-bit2022.06 | 4 | |
| AQ-SGDNetwork Bandwidth=10 Gbps, Forward quantization (fw)=3-bit, Backward quantization (bw)=6-bit2022.06 | 4 | |
| AQ-SGDNetwork Bandwidth=10 Gbps, Forward quantization (fw)=4-bit, Backward quantization (bw)=8-bit2022.06 | 4 | |
| AQ-SGDNetwork Bandwidth=1 Gbps, Forward quantization (fw)=3-bit, Backward quantization (bw)=6-bit2022.06 | 4 | |
| DirectQNetwork Bandwidth=500 Mbps, Forward quantization (fw)=3-bit, Backward quantization (bw)=6-bit2022.06 | 3.9 | |
| DirectQNetwork Bandwidth=500 Mbps, Forward quantization (fw)=4-bit, Backward quantization (bw)=8-bit2022.06 | 3.9 | |
| DirectQNetwork Bandwidth=300 Mbps, Forward quantization (fw)=3-bit, Backward quantization (bw)=6-bit2022.06 | 3.9 | |
| AQ-SGDNetwork Bandwidth=1 Gbps, Forward quantization (fw)=4-bit, Backward quantization (bw)=8-bit2022.06 | 3.9 | |
| AQ-SGDNetwork Bandwidth=500 Mbps, Forward quantization (fw)=3-bit, Backward quantization (bw)=6-bit2022.06 | 3.9 | |
| AQ-SGDNetwork Bandwidth=500 Mbps, Forward quantization (fw)=4-bit, Backward quantization (bw)=8-bit2022.06 | 3.9 | |
| FP32Network Bandwidth=10 Gbps, Precision=Full (FP32)2022.06 | 3.8 | |
| DirectQNetwork Bandwidth=300 Mbps, Forward quantization (fw)=4-bit, Backward quantization (bw)=8-bit2022.06 | 3.8 | |
| AQ-SGDNetwork Bandwidth=300 Mbps, Forward quantization (fw)=3-bit, Backward quantization (bw)=6-bit2022.06 | 3.8 | |
| AQ-SGDNetwork Bandwidth=300 Mbps, Forward quantization (fw)=4-bit, Backward quantization (bw)=8-bit2022.06 | 3.8 | |
| DirectQNetwork Bandwidth=100 Mbps, Forward quantization (fw)=3-bit, Backward quantization (bw)=6-bit2022.06 | 3.5 | |
| AQ-SGDNetwork Bandwidth=100 Mbps, Forward quantization (fw)=3-bit, Backward quantization (bw)=6-bit2022.06 | 3.4 | |
| FP32Network Bandwidth=1 Gbps, Precision=Full (FP32)2022.06 | 3.2 | |
| DirectQNetwork Bandwidth=100 Mbps, Forward quantization (fw)=4-bit, Backward quantization (bw)=8-bit2022.06 | 3 | |
| AQ-SGDNetwork Bandwidth=100 Mbps, Forward quantization (fw)=4-bit, Backward quantization (bw)=8-bit2022.06 | 3 | |
| FP32Network Bandwidth=500 Mbps, Precision=Full (FP32)2022.06 | 2.7 | |
| FP32Network Bandwidth=300 Mbps, Precision=Full (FP32)2022.06 | 1.8 | |
| FP32Network Bandwidth=100 Mbps, Precision=Full (FP32)2022.06 | 0.5 |