Language Modeling on GPT-2 Medium (val)
87.89RuntimePACI
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| PACIBatch size=256, Num. micro-batches=162026.06 | 87.89 | 15.291 | 1.41 | 1.41 | |
| PACIBatch size=128, Num. micro-batches=82026.06 | 87.99 | 15.483 | 1.84 | 1.69 | |
| PACIBatch size=128, Num. micro-batches=42026.06 | 111.51 | 15.632 | 2.04 | 1.33 | |
| PACIBatch size=256, Num. micro-batches=82026.06 | 111.91 | 15.399 | 1.51 | 1.11 | |
| FlushBatch size=256, Num. micro-batches=162026.06 | 124.27 | 15.35 | 1 | 1 | |
| FlushBatch size=128, Num. micro-batches=162026.06 | 148.64 | 15.48 | 1 | 1 | |
| FlushBatch size=128, Num. micro-batches=82026.06 | 162.33 | 15.48 | 1 | 0.92 | |
| FlushBatch size=256, Num. micro-batches=82026.06 | 169.38 | 15.35 | 1 | 0.73 | |
| FlushBatch size=128, Num. micro-batches=42026.06 | 227.62 | 15.48 | 1 | 0.65 |