Generation on Big-Bench Hard (test)
57.9Exact MatchFLAN-PaLM 540B
Evaluation results
| Method | Links | |
|---|---|---|
| FLAN-PaLM 540B# shots=32022.12 | 57.9 | |
| OpenAI code-davinci-002# shots=32022.12 | 52.8 | |
| OpenAI text-davinci-003# shots=32022.12 | 50.9 | |
| PaLM 540B# shots=32022.12 | 49.1 | |
| OpenAI text-davinci-002# shots=32022.12 | 48.6 | |
| FLAN-PaLM 62B# shots=32022.12 | 47.5 | |
| FLAN-T5 11B# shots=32022.12 | 45.3 | |
| PaLM 62B# shots=32022.12 | 37.4 | |
| OPT-IML-Max 175B# shots=32022.12 | 35.7 | |
| OpenAI davinci# shots=32022.12 | 33.6 | |
| OPT-IML-Max 30B# shots=32022.12 | 30.9 | |
| OPT 175B# shots=32022.12 | 30.2 | |
| T5 11B# shots=32022.12 | 29.5 | |
| OPT 30B# shots=32022.12 | 28.4 | |
| OPT 1.3B# shots=32022.12 | 27.9 | |
| OPT-IML-Max 1.3B# shots=32022.12 | 26.5 | |
| T0pp 11B# shots=32022.12 | 13 |