Masked Language Modeling on BERT MLM small (val)
6.9412Validation LossSE-GEM
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| SE-GEMepsilon=10, Steps=3,000, Seeds=3, tok/s=20,7012026.04 | 6.9412 | 0.035 | 39.99 | 0.253 |
Loading the SOTA2 catalog…
| Method | Links | ||||
|---|---|---|---|---|---|
| SE-GEMepsilon=10, Steps=3,000, Seeds=3, tok/s=20,7012026.04 | 6.9412 | 0.035 | 39.99 | 0.253 |