Reasoning and Language Understanding on BigBench Emergent Suite (BBES)
67NavigateU-PaLM 540B
Evaluation results
| Method | Links | ||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| U-PaLM 540BPrompting=5-shot, Number of parameters=540B2022.10 | 67 | 78.3 | 100 | 86.5 | 73.9 | 78.4 | 87.5 | 59.9 | 68.4 | 87.5 | 49.3 | 12.5 | 86.1 | 37.5 | 68 | 68.8 | 81.2 | 59.8 | 81.4 | 51 | 38.9 | 67.7 | |
| PaLM 540BPrompting=5-shot, Number of parameters=540B2022.10 | 55.3 | 73.9 | 97.7 | 92.3 | 70.2 | 75.7 | 87.2 | 54.2 | 65.3 | 91.2 | 44 | 7.6 | 69.1 | 36.5 | 65.9 | 64.8 | 78.3 | 54.8 | 80.3 | 47 | 36 | 64.3 |