Next-token prediction on Wikipedia English (test)
640Bits per Token (BPT)ASTRA
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| ASTRA#Groups=1, Quantization=8-bit, Number of devices=4, Input token length=10242025.05 | 640 | 7.7336 | 1,638.4 | |
| ASTRA#Groups=16, Quantization=8-bit, Number of devices=4, Input token length=10242025.05 | 10,240 | 7.5879 | 102.4 | |
| ASTRA#Groups=32, Quantization=8-bit, Number of devices=4, Input token length=10242025.05 | 20,480 | 7.436 | 51.2 | |
| Llama-3-8BQuantization=8-bit, Number of devices=4, Input token length=10242025.05 | 1,048,576 | 5.8118 | — |