Task Generalization on T0 Taxonomy Evaluation Tasks (val)
59.1OBQAT0 11B
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| T0 11Bsemi-param=No, # train tasks=36, # param=11B2022.10 | 59.1 | 72.5 | 81.8 | 70.1 | 91.5 | 55.2 | 33.5 | 58.1 | 66.3 | |
| GPT-3semi-param=No, # param=175B, Protocol=Few-shot (32 examples)2022.10 | 57.6 | 81 | 59.7 | 46.4 | 91 | 49.4 | 78.9 | 62.7 | 66.3 | |
| ZemiLARGEsemi-param=Yes, # train tasks=8, # param=0.8B2022.10 | 51.5 | 67.9 | 84.1 | 62.1 | 84.5 | 50.4 | 35.8 | 53.5 | 62.3 | |
| T0 3Bsemi-param=No, # train tasks=36, # param=3B2022.10 | 42.8 | 59.3 | 73.6 | 45.5 | 75.9 | 50 | 27.3 | 45 | 53.5 | |
| ReCrosssemi-param=Yes, # train tasks=36, # param=0.4B2022.10 | 39.6 | 41.4 | — | 44.8 | — | 50.6 | 47.3 | 44.7 | — | |
| ZemiBASEsemi-param=Yes, # train tasks=8, # param=0.2B2022.10 | 35.6 | 59.2 | 68.6 | 50.1 | 63.6 | 49.6 | 29.7 | 44.8 | 50.9 | |
| BARTOsemi-param=No, # train tasks=36, # param=0.4B2022.10 | 34.4 | 36.1 | — | 39.6 | — | 46.7 | 39.4 | 39.3 | — |