Zero-shot cross-task generalization on Super-NaturalInstructions (test)
34.97ROUGE-LDef. + Instruction Embedder
Evaluation results
| Method | Links | |
|---|---|---|
| Def. + Instruction EmbedderOptimization=Instance Conversion (IC), Training Instruction=Def. + learnable instruction2023.06 | 34.97 | |
| Def. + Instruction EmbedderOptimization=Direct Parameterization (DP), Training Instruction=Def. + learnable instruction2023.06 | 34.79 | |
| Def.Training Instruction=Task definition2023.06 | 33.82 | |
| Def. + Pos. + Neg. + Expl.Training Instruction=Task definition + 1 positive/negative exemplar + explanation2023.06 | 29.07 | |
| Def. + Pos. + Neg.Training Instruction=Task definition + 1 positive exemplar + 1 negative exemplar2023.06 | 27.91 | |
| Def. + Pos.Training Instruction=Task definition + 1 positive exemplar2023.06 | 27.74 | |
| Instruction EmbedderOptimization=Instance Conversion (IC)2023.06 | 11.99 | |
| Instruction EmbedderOptimization=Direct Parameterization (DP)2023.06 | 11.79 |