Zero-shot task generalization on NI (Super-NaturalInstructions) v2 (test)
56.12ROUGE-LTk-INSTRUCT-XL + triplet + meta
Evaluation results
| Method | Links | |
|---|---|---|
| Tk-INSTRUCT-XL + triplet + metaNumber of Parameters=3B, Definition Strategy=triplet-structured + meta-tuning, Evaluation Protocol=zero-shot2023.06 | 56.12 | |
| Tk-INSTRUCT-XL + tripletNumber of Parameters=3B, Definition Strategy=triplet-structured, Evaluation Protocol=zero-shot2023.06 | 55.58 | |
| Tk-INSTRUCT-XLNumber of Parameters=3B, Definition Strategy=standard, Evaluation Protocol=zero-shot2023.06 | 54.08 | |
| InstructGPTNumber of Parameters=175B, Evaluation Protocol=zero-shot2023.06 | 52.1 | |
| Tk-INSTRUCT-Large + triplet + metaNumber of Parameters=770M, Definition Strategy=triplet-structured + meta-tuning, Evaluation Protocol=zero-shot2023.06 | 51.46 | |
| Tk-INSTRUCT-Large + tripletNumber of Parameters=770M, Definition Strategy=triplet-structured, Evaluation Protocol=zero-shot2023.06 | 50.84 | |
| Tk-INSTRUCT-LargeNumber of Parameters=770M, Definition Strategy=standard, Evaluation Protocol=zero-shot2023.06 | 47.5 | |
| BART-Large + triplet + metaNumber of Parameters=340M, Definition Strategy=triplet-structured + meta-tuning, Evaluation Protocol=zero-shot2023.06 | 44.89 | |
| BART-Large + tripletNumber of Parameters=340M, Definition Strategy=triplet-structured, Evaluation Protocol=zero-shot2023.06 | 43.76 | |
| BART-LargeNumber of Parameters=340M, Definition Strategy=full definition, Evaluation Protocol=zero-shot2023.06 | 40.7 | |
| HeuristicsEvaluation Protocol=zero-shot2023.06 | 38.61 | |
| T0Number of Parameters=11B, Evaluation Protocol=zero-shot2023.06 | 32.3 |