Masked Language Modeling on XSUM randomly sampled
3.8U-PPLMLM
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| MLMBackbone=BERT-large, Masking Type=Random pairs2023.05 | 3.8 | 5.67 | 0.53 | 0.413 | |
| MRFBackbone=BERT-large, Masking Type=Random pairs2023.05 | 3.87 | 4.94 | 0.322 | 0.036 | |
| AGBackbone=BERT-large, Masking Type=Random pairs2023.05 | 3.88 | 4.13 | 0.003 | 0.042 | |
| HCBBackbone=BERT-large, Masking Type=Random pairs2023.05 | 3.91 | 5.14 | 0.346 | 0.059 | |
| MLMBackbone=BERT-large, Masking Type=Contiguous pairs2023.05 | 3.91 | 103.86 | 5.046 | 3.276 | |
| MRFLBackbone=BERT-large, Masking Type=Random pairs2023.05 | 3.94 | 7.06 | 0.156 | 0.068 | |
| MRFBackbone=BERT-large, Masking Type=Contiguous pairs2023.05 | 4.16 | 36.66 | 2.258 | 0.145 | |
| AGBackbone=BERT-large, Masking Type=Contiguous pairs2023.05 | 4.21 | 6.62 | 0.016 | 0.126 | |
| MRFLBackbone=BERT-large, Masking Type=Contiguous pairs2023.05 | 4.62 | 1,328.2 | 1.441 | 0.29 | |
| MLMBackbone=BERT-base, Masking Type=Random pairs2023.05 | 4.88 | 6.12 | 0.404 | 0.227 | |
| MLMBackbone=BERT-base, Masking Type=Contiguous pairs2023.05 | 4.91 | 39.33 | 4.381 | 2.128 | |
| MRFBackbone=BERT-base, Masking Type=Random pairs2023.05 | 5 | 6.23 | 0.262 | 0.049 | |
| AGBackbone=BERT-base, Masking Type=Random pairs2023.05 | 5 | 5.29 | 0.003 | 0.044 | |
| HCBBackbone=BERT-base, Masking Type=Random pairs2023.05 | 5.08 | 6.21 | 0.256 | 0.052 | |
| MRFLBackbone=BERT-base, Masking Type=Random pairs2023.05 | 5.17 | 9.12 | 0.148 | 0.085 | |
| AGBackbone=BERT-base, Masking Type=Contiguous pairs2023.05 | 5.27 | 8.42 | 0.016 | 0.143 | |
| MRFBackbone=BERT-base, Masking Type=Contiguous pairs2023.05 | 5.53 | 47.62 | 2.242 | 0.185 | |
| HCBBackbone=BERT-large, Masking Type=Contiguous pairs2023.05 | 5.67 | 164.15 | 2.954 | 0.4 | |
| HCBBackbone=BERT-base, Masking Type=Contiguous pairs2023.05 | 6.46 | 174.32 | 2.681 | 0.328 | |
| MRFLBackbone=BERT-base, Masking Type=Contiguous pairs2023.05 | 6.55 | 2,209.94 | 1.561 | 0.383 |