BIG-bench (Multi-task Language Understanding)
80Hindu KnowledgeGopher
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| GopherNumber of Parameters=280B, Evaluation Protocol=5-Shot2021.12 | 80 | 63.6 | 61.7 | 59.1 | 68.6 | |
| GopherNumber of Parameters=280B, Evaluation Protocol=Zero-Shot2021.12 | 56.97 | 65.91 | 55.98 | 22.73 | 50.4 | |
| T0++Evaluation Protocol=Zero-Shot2021.12 | 40 | 52.17 | 54.79 | 28.12 | 43.77 | |
| T0+Evaluation Protocol=Zero-Shot2021.12 | 38.29 | 63.04 | 52.97 | 31.25 | 46.39 | |
| GopherNumber of Parameters=7.1B, Evaluation Protocol=Zero-Shot2021.12 | 37.58 | 59.09 | 51.2 | 18.18 | 41.51 | |
| T0Evaluation Protocol=Zero-Shot2021.12 | 36 | 63.04 | 52.51 | 15.62 | 41.79 | |
| GopherNumber of Parameters=117M, Evaluation Protocol=Zero-Shot2021.12 | 32.12 | 36.36 | 46.89 | 0 | 28.84 | |
| GopherNumber of Parameters=1B, Evaluation Protocol=Zero-Shot2021.12 | 32.12 | 54.55 | 47.37 | 9.09 | 35.78 | |
| GopherNumber of Parameters=44M, Evaluation Protocol=Zero-Shot2021.12 | 26.06 | 47.73 | 49.76 | 4.55 | 32.03 | |
| GopherNumber of Parameters=417M, Evaluation Protocol=Zero-Shot2021.12 | 24.85 | 47.73 | 48.8 | 4.55 | 31.48 |