Binary Classification on AdvGLUE (test)
0.701QNLI AccuracyDropout
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Dropoutbackbone=DeBERTa-large, scale=350M, paradigm=Self-trained, labels=none2023.05 | 0.701 | 0.633 | 0.709 | 0.499 | 0.636 | |
| SimPLEbackbone=DeBERTa-large, scale=350M, paradigm=Self-trained, labels=none2023.05 | 0.701 | 0.681 | 0.738 | 0.516 | 0.659 | |
| SETREDbackbone=DeBERTa-large, scale=350M, paradigm=Self-trained, labels=none2023.05 | 0.698 | 0.695 | 0.699 | 0.509 | 0.65 | |
| SimPLEbackbone=RoBERTa-large, scale=350M, paradigm=Self-trained, labels=none2023.05 | 0.696 | 0.544 | 0.623 | 0.588 | 0.613 | |
| Dropoutbackbone=RoBERTa-large, scale=350M, paradigm=Self-trained, labels=none2023.05 | 0.692 | 0.578 | 0.619 | 0.573 | 0.616 | |
| SETREDbackbone=RoBERTa-large, scale=350M, paradigm=Self-trained, labels=none2023.05 | 0.68 | 0.565 | 0.626 | 0.589 | 0.615 | |
| Baseline-STbackbone=DeBERTa-large, scale=350M, paradigm=Self-trained, labels=none2023.05 | 0.658 | 0.704 | 0.684 | 0.509 | 0.639 | |
| Baseline-STbackbone=RoBERTa-large, scale=350M, paradigm=Self-trained, labels=none2023.05 | 0.649 | 0.606 | 0.609 | 0.566 | 0.608 | |
| RoBERTa-Supparadigm=Zero-shot, scale=350M, adaptation=Supposition classification2023.05 | 0.621 | 0.526 | 0.617 | 0.599 | 0.591 | |
| DeBERTa-Supparadigm=Zero-shot, scale=350M, adaptation=Supposition classification2023.05 | 0.615 | 0.641 | 0.667 | 0.426 | 0.587 | |
| DeBERTa-Catparadigm=Zero-shot, scale=350M, adaptation=Concatenation-based2023.05 | 0.608 | 0.474 | 0.506 | 0.561 | 0.537 | |
| DeBERTaparadigm=Fully-supervised, scale=350M, labels=human-generated2023.05 | 0.579 | 0.604 | 0.79 | 0.578 | 0.638 | |
| RoBERTaparadigm=Fully-supervised, scale=350M, labels=human-generated2023.05 | 0.525 | 0.454 | 0.628 | 0.585 | 0.548 | |
| CTparadigm=Fully-supervised, scale=350M, labels=human-generated2023.05 | 0.496 | 0.407 | 0.462 | 0.392 | 0.439 | |
| R3Fparadigm=Fully-supervised, scale=350M, labels=human-generated2023.05 | 0.475 | 0.406 | 0.501 | 0.385 | 0.442 | |
| MTparadigm=Fully-supervised, scale=350M, labels=human-generated2023.05 | 0.475 | 0.415 | 0.525 | 0.513 | 0.482 | |
| BERTparadigm=Fully-supervised, scale=350M, labels=human-generated2023.05 | 0.398 | 0.379 | 0.405 | 0.33 | 0.378 |