Natural Language Understanding on GLUE (MNLI-m, QNLI, SST-2, MRPC)
87.6MNLI-m AccuracyRoBERTa_BASE
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| RoBERTa_BASEPre-train (P)=true, Pre-train (D)=false, Pre-train (W)=false2022.10 | 87.6 | 92.8 | 94.8 | 90.2 | |
| XDocPre-training Steps=100K, Pre-train (P)=true, Pre-train (D)=true, Pre-train (W)=false2022.10 | 87.2 | 92.7 | 94.9 | 90.2 | |
| XDocPre-training Steps=100K, Pre-train (P)=true, Pre-train (D)=false, Pre-train (W)=false2022.10 | 87 | 93 | 95.2 | 90.1 | |
| XDocPre-training Steps=100K, Pre-train (P)=true, Pre-train (D)=false, Pre-train (W)=true2022.10 | 86.8 | 92.3 | 95.1 | 90.6 | |
| XDocPre-training Steps=1M, Pre-train (P)=true, Pre-train (D)=true, Pre-train (W)=true2022.10 | 86.8 | 92.3 | 95.3 | 91.1 | |
| XDocPre-training Steps=100K, Pre-train (P)=false, Pre-train (D)=true, Pre-train (W)=false2022.10 | 86.7 | 91.3 | 94.5 | 89.9 | |
| XDocPre-training Steps=500K, Pre-train (P)=true, Pre-train (D)=true, Pre-train (W)=true2022.10 | 86.6 | 92.2 | 95.2 | 89.9 | |
| XDocPre-training Steps=100K, Pre-train (P)=false, Pre-train (D)=false, Pre-train (W)=true2022.10 | 86.5 | 92 | 94.6 | 90.1 | |
| XDocPre-training Steps=100K, Pre-train (P)=false, Pre-train (D)=true, Pre-train (W)=true2022.10 | 86.4 | 91.6 | 95.3 | 91 | |
| XDocPre-training Steps=100K, Pre-train (P)=true, Pre-train (D)=true, Pre-train (W)=true2022.10 | 86.2 | 92.8 | 95.2 | 91.3 |