Specificity Testing on S 1.0 (test)
89Birthplace AccuracyRoBERTa-Large (CP)
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| RoBERTa-Large (CP)Backbone=RoBERTa-Large, Prompting Technique=Cascade Prompting2022.10 | 89 | 74.02 | 66.09 | 79.87 | 65.18 | 74.83 | |
| BERT-Large (CP)Backbone=BERT-Large, Prompting Technique=Cascade Prompting2022.10 | 82.25 | 70.02 | 53.55 | 77.67 | 71.88 | 71.07 | |
| GPT-2 (FP)Backbone=GPT-2, Prompting Technique=Few-shot Prompting, Number of demonstrations (K)=102022.10 | 81.01 | 71.66 | 50.33 | 64.15 | 57.67 | 64.96 | |
| RoBERTa-Large (FP)Backbone=RoBERTa-Large, Prompting Technique=Few-shot Prompting, Number of demonstrations (K)=102022.10 | 70.51 | 71.94 | 42.26 | 73.7 | 62.94 | 64.27 | |
| BERT-Base (FP)Backbone=BERT-Base, Prompting Technique=Few-shot Prompting, Number of demonstrations (K)=102022.10 | 67.85 | 70.54 | 50.11 | 69.11 | 53.83 | 62.29 | |
| BERT-Large (FP)Backbone=BERT-Large, Prompting Technique=Few-shot Prompting, Number of demonstrations (K)=102022.10 | 66.17 | 64.7 | 50.37 | 65.44 | 52.24 | 59.78 | |
| RoBERTa-Base (FP)Backbone=RoBERTa-Base, Prompting Technique=Few-shot Prompting, Number of demonstrations (K)=102022.10 | 64.85 | 72.38 | 35.85 | 63.01 | 51.11 | 57.44 | |
| RoBERTa-Base (CP)Backbone=RoBERTa-Base, Prompting Technique=Cascade Prompting2022.10 | 63.09 | 64.54 | 54.56 | 61.81 | 62.78 | 61.36 | |
| BERT-Base (VP)Backbone=BERT-Base, Prompting Technique=Vanilla Prompting2022.10 | 60.68 | 70.46 | 49.09 | 67.64 | 67.41 | 63.06 | |
| GPT-2 (VP)Backbone=GPT-2, Prompting Technique=Vanilla Prompting2022.10 | 59.72 | 57.28 | 48.25 | 57.98 | 60.86 | 56.82 | |
| GPT-2 (CP)*Backbone=GPT-2, Prompting Technique=Cascade Prompting, Rescore all suffixes=false2022.10 | 59.72 | 57.28 | 48.25 | 57.98 | 60.86 | 56.82 | |
| BERT-Base (CP)Backbone=BERT-Base, Prompting Technique=Cascade Prompting2022.10 | 59.68 | 70.54 | 55.06 | 67.42 | 69.49 | 64.44 | |
| BERT-Large (VP)Backbone=BERT-Large, Prompting Technique=Vanilla Prompting2022.10 | 56.52 | 71.76 | 42.36 | 77.25 | 66.77 | 62.93 | |
| RoBERTa-Base (VP)Backbone=RoBERTa-Base, Prompting Technique=Vanilla Prompting2022.10 | 54.48 | 61.8 | 49.99 | 61.59 | 59.11 | 57.39 | |
| RoBERTa-Large (VP)Backbone=RoBERTa-Large, Prompting Technique=Vanilla Prompting2022.10 | 42.16 | 71.44 | 43.28 | 80.63 | 59.27 | 59.36 |