Language Model Pre-training on BERT-Large NVIDIA 2080 Ti (train)
50Max Batch SizeCheckpoint
Evaluation Results
| Method | Links | |
|---|---|---|
| CheckpointSequence Length=1282022.10 | 50 | |
| TempoSequence Length=1282022.10 | 24 | |
| BaselineSequence Length=1282022.10 | 15 | |
| CheckpointSequence Length=5122022.10 | 4 | |
| TempoSequence Length=5122022.10 | 2 | |
| BaselineSequence Length=5122022.10 | 1 |