Language Model Pre-training on BERT-Large NVIDIA V100 (train)
96Max Batch SizeCheckpoint
Evaluation results
| Method | Links | |
|---|---|---|
| CheckpointSequence Length=1282022.10 | 96 | |
| TempoSequence Length=1282022.10 | 41 | |
| BaselineSequence Length=1282022.10 | 28 | |
| CheckpointSequence Length=5122022.10 | 18 | |
| TempoSequence Length=5122022.10 | 7 | |
| BaselineSequence Length=5122022.10 | 4 |