Predicting Object Attributes on PIGPEN-NLU overall (test)
81.1AccuracyPIGLET
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| PIGLET2021.06 | 81.1 | 92.3 | 91.9 | 99.2 | 99.8 | 99 | |
| T5-11BApproach category=text-to-text2021.06 | 64.2 | 83.9 | 88.9 | 94.3 | 95.4 | 98.1 | |
| T5-3BApproach category=text-to-text2021.06 | 63.3 | 81.6 | 90 | 94 | 95.6 | 98.4 | |
| T5-LargeApproach category=text-to-text2021.06 | 54.1 | 81.8 | 84.6 | 94.3 | 96.3 | 95.8 | |
| T5-BaseApproach category=text-to-text2021.06 | 53.9 | 81.1 | 87.5 | 93.6 | 96.1 | 96.5 | |
| Alberti et al.2019Approach category=BERT-style, Dynamics pre-training=true2021.06 | 53.9 | 87.7 | 87.6 | 97.5 | 93.4 | 97.5 | |
| T5-SmallApproach category=text-to-text2021.06 | 36.2 | 82.2 | 84.9 | 93.8 | 89.6 | 93.5 | |
| G&D2019Approach category=BERT-style, Dynamics pre-training=true2021.06 | 35.3 | 83 | 86.9 | 94 | 93.7 | 97.4 | |
| No Change2021.06 | 25.5 | 83.2 | 84.1 | 96.3 | 86 | 94.8 | |
| GPT3-175Bfew-shot=true2021.06 | 22.4 | 73.7 | 77 | 89.5 | 84.2 | 94.7 | |
| G&D2019Approach category=BERT-style, Dynamics pre-training=false2021.06 | 11.3 | 68.6 | 47.3 | 82.2 | 88.3 | 95.8 | |
| Alberti et al.2019Approach category=BERT-style, Dynamics pre-training=false2021.06 | 6.8 | 53.4 | 43.6 | 84 | 88.1 | 95.1 |