Paraphrase Identification on PAWS-X (test) (Multi-lingual)
94Accuracy (en)mBERT (Reference [1])
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| mBERT (Reference [1])Reference=[1]2021.06 | 94 | 85.7 | 87.4 | 87 | 77 | 69.6 | 73 | 82 | — | |
| Syntax-augmented mBERTsyntax-augmented=true2021.06 | 94 | 85.9 | 89.1 | 88.2 | 80.7 | 76.3 | 75.8 | 84.3 | — | |
| mBERTbaseline=true2021.06 | 93.9 | 85.7 | 88.4 | 87.6 | 78 | 73.6 | 73.1 | 82.9 | — | |
| DPAShots=512, Method_Variant=Dual Prompt Augmentation2022.02 | 84.97 | 78.63 | 79.6 | 80.48 | 72.34 | 68.13 | 67.86 | 76 | — | |
| DPA W/O MIXUPShots=512, Mixup=false2022.02 | 84.84 | 77.85 | 79.36 | 79.69 | 71.03 | 68.03 | 66.76 | 75.37 | — | |
| DPA w/o MVShots=512, Multilingual Verbalizer=false2022.02 | 84.81 | 78.56 | 79.67 | 79.64 | 71.5 | 68.34 | 67.04 | 75.65 | — | |
| UPShots=512, Method_Variant=Universal Prompting2022.02 | 83.31 | 76.18 | 77.63 | 77.42 | 68.06 | 65.03 | 63.41 | 73.01 | — | |
| FTShots=512, Protocol=finetuning2022.02 | 77.64 | 73.41 | 73.19 | 74.33 | 68.25 | 65.19 | 65.55 | 71.08 | — | |
| DPAShots=256, Method_Variant=Dual Prompt Augmentation2022.02 | 71.87 | 68.59 | 69.1 | 69.02 | 62.75 | 60.88 | 60.41 | 66.09 | — | |
| DPA W/O MIXUPShots=256, Mixup=false2022.02 | 70.95 | 67.14 | 67.58 | 67.63 | 61.16 | 60.44 | 59.01 | 64.84 | — | |
| DPA w/o MVShots=256, Multilingual Verbalizer=false2022.02 | 69.06 | 66.26 | 66.47 | 65.79 | 60.77 | 58.34 | 59.28 | 63.71 | — | |
| UPShots=256, Method_Variant=Universal Prompting2022.02 | 65.5 | 62.21 | 63.24 | 62.82 | 55.99 | 54.3 | 54.11 | 59.74 | — | |
| FTShots=256, Protocol=finetuning2022.02 | 63.18 | 60.81 | 60.95 | 61.39 | 59.78 | 58.48 | 58.6 | 60.46 | — | |
| BXLLaMA2024.06 | — | — | — | — | — | — | — | — | 58.77 | |
| LLaMA2024.06 | — | — | — | — | — | — | — | — | 60.56 | |
| SFTLLaMA2024.06 | — | — | — | — | — | — | — | — | 60.63 | |
| xLLMs-1002024.06 | — | — | — | — | — | — | — | — | 66.43 |