Instruction Following on Super-NaturalInstructions v2 (held-out categories)
64.1Textual Entailment ROUGE-L F1Tk-Instruct 11B
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Tk-Instruct 11Bshots=2, mode=generation, training=instructions + 2 positive demonstrations2022.12 | 64.1 | 62.3 | 69.6 | 65.3 | |
| Tk-Instruct 11Bshots=5, mode=generation, training=instructions + 2 positive demonstrations2022.12 | 62.3 | 57.1 | 55.8 | 58.4 | |
| Tk-Instruct 11Bshots=0, mode=generation, training=instructions + 2 positive demonstrations2022.12 | 55 | 32.3 | 51.1 | 46.1 | |
| OPT-IML 30Bshots=0, mode=generation2022.12 | 54.7 | 37.4 | 51.4 | 47.9 | |
| OPT-IML 175Bshots=0, mode=generation2022.12 | 54.3 | 39 | 61.2 | 51.5 | |
| OPT-IML 175Bshots=5, mode=generation2022.12 | 51.5 | 50.9 | 56.5 | 53 | |
| OPT-IML 175Bshots=2, mode=generation2022.12 | 51 | 49.8 | 60.2 | 53.6 | |
| OPT-IML 30Bshots=5, mode=generation2022.12 | 49.8 | 43.8 | 47.2 | 46.9 | |
| OPT-IML 30Bshots=2, mode=generation2022.12 | 47.8 | 41.6 | 51.8 | 47.1 | |
| OPT 175Bshots=5, mode=generation2022.12 | 43.6 | 43.6 | 48.2 | 45.2 | |
| OPT 30Bshots=5, mode=generation2022.12 | 42.7 | 43.4 | 48.2 | 44.8 | |
| OPT 175Bshots=0, mode=generation2022.12 | 41.6 | 21 | 37.1 | 33.3 | |
| OPT 30Bshots=0, mode=generation2022.12 | 40.3 | 21.3 | 35.2 | 32.3 | |
| OPT 175Bshots=2, mode=generation2022.12 | 2.2 | 4.2 | 16.8 | 7.7 | |
| OPT 30Bshots=2, mode=generation2022.12 | 0.9 | 1.1 | 4.1 | 2 |