Natural Language Processing on T0 benchmark
85.8RTET0*
Evaluation Results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| T0*Base Model=T5-XXL, #Params=11B2022.11 | 85.8 | 73.3 | 47.3 | 42 | 46.1 | 94.4 | 31.5 | 98.4 | 62.8 | 72.8 | 56 | 64.6 | |
| UDBase Model=T5-Large, #Params=400M2022.11 | 83.8 | 80.4 | 36.8 | 34.2 | 42.2 | 90 | 56.1 | 96.4 | 68.3 | 62.9 | 54.6 | 64.1 | |
| UD+Base Model=T5-XXL, #Params=5.5B2022.11 | 82 | 89.3 | 53.4 | 48.1 | 51 | 96 | 78.9 | 96.7 | 75 | 86.4 | 58.5 | 74.1 | |
| T0†Base Model=T5-XXL, #Params=11B2022.11 | 80.8 | 70.1 | 43.6 | 38.7 | 41.3 | 90 | 33.6 | 92.4 | 61.5 | 59.9 | 56.6 | 60.8 | |
| UDBase Model=T5-XXL, #Params=5.5B2022.11 | 80.5 | 87.5 | 49 | 42.9 | 48.8 | 95 | 77.4 | 98.6 | 73.1 | 82.2 | 57.1 | 72 | |
| T0*Base Model=T5-XL, #Params=3B2022.11 | 79.7 | 68.9 | 43.1 | 38.5 | 42.3 | 94.1 | 31.5 | 97.5 | 68.8 | 61.3 | 54.1 | 61.8 | |
| UDBase Model=T5-XL, #Params=1.5B2022.11 | 78.7 | 73.2 | 41.2 | 36.3 | 45.4 | 94 | 70.1 | 97.9 | 72.1 | 70.6 | 53 | 66.6 | |
| FLANBase Model=Decoder-only, #Params=137B2022.11 | 78.3 | 64.1 | 47.7 | 43.9 | 47 | 90.6 | 56.4 | 92.2 | 80.8 | 67.3 | — | — | |
| T0*Base Model=T5-Large, #Params=800M2022.11 | 75.1 | 55.5 | 32.9 | 32.3 | 33.7 | 84.6 | 28.2 | 94 | 63 | 54.6 | 51.2 | 55 | |
| PaLMBase Model=Decoder-only, #Params=540B2022.11 | 72.9 | 51.8 | 48 | 44.2 | 45.7 | 93 | 83.4 | 84.6 | 89.1 | 81.1 | 59.1 | 68.5 | |
| UDBase Model=DeBERTaV3, #Params=304M2022.11 | 71.1 | 76.8 | 43.8 | 41.3 | 45.7 | 96 | 60.7 | 97.4 | 66.4 | 83.6 | 53.3 | 66.9 | |
| PE-REPBase Model=ELECTRA, #Params=335M2022.11 | 69 | 61.3 | 36.1 | 35 | 39.4 | 91.2 | 47 | 96.8 | 70 | 56.2 | 51.1 | 58.5 | |
| GLaMBase Model=MoE Decoder-only, #Params=64B2022.11 | 66.8 | 33.9 | 40.9 | 38.2 | 40.9 | 90 | 77.1 | 82.5 | 83.5 | 73.4 | 50.5 | 61.6 | |
| T0†Base Model=T5-XL, #Params=3B2022.11 | 64.6 | 45.4 | 33.8 | 33.1 | 33.3 | 72.4 | 27.3 | 84 | 65.1 | 51 | 50.7 | 51 | |
| GPT-3Base Model=Decoder-only, #Params=175B2022.11 | 63.5 | 46.4 | 34.6 | 35.4 | 34.5 | 91 | 78.9 | 83.2 | 65.4 | 70.2 | — | — | |
| PE-CLSBase Model=ELECTRA, #Params=335M2022.11 | 60.2 | 57.4 | 34.1 | 34.4 | 36.4 | 92.7 | 44.1 | 96 | 62.8 | 56.3 | 50.7 | 56.8 | |
| GLaMBase Model=Decoder-only, #Params=137B2022.11 | 56.3 | 39.3 | 39.7 | 35.5 | 34.1 | 90 | 76.7 | 81.1 | 82.1 | 71.3 | 50.6 | 59.7 | |
| PE-PROBBase Model=ELECTRA, #Params=335M2022.11 | 54 | 49.2 | 32.3 | 33.3 | 33.5 | 81.9 | 36.7 | 89.5 | 64.3 | 50.7 | 50.9 | 52.4 |