# Few-Shot Self-Rationalization with Natural Language Prompts Source: [Few-Shot Self-Rationalization with Natural Language Prompts]() Collection index: [llms.txt]() Data updated: 2025-10-26T00:09:48.320Z Authors: Ana Marasovi\\'c, Iz Beltagy, Doug Downey, Matthew E. Peters Published: 2021-11-16T08:21:40.000Z ## Abstract Self-rationalization models that predict task labels and generate free-text elaborations for their predictions could enable more intuitive interaction with NLP systems. These models are, however, currently trained with a large amount of human-written free-text explanations for each task which hinders their broader usage. We propose to study a more realistic setting of self-rationalization using few training examples. We present FEB -- a standardized collection of four existing English-language datasets and associated metrics. We identify the right prompting approach by extensively exploring natural language prompts on FEB. Then, by using this prompt and scaling the model size, we demonstrate that making progress on few-shot self-rationalization is possible. We show there is still ample room for improvement in this task: the average plausibility of generated explanations assessed by human annotators is at most 51% (with GPT-3), while plausibility of human explanations is 76%. We hope that FEB and our proposed approach will spur the community to take on the few-shot self-rationalization challenge. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Natural Language Explanation Generation on SBIC 60-shot](): Natural Language Explanation Generation; SBIC 60-shot; Accuracy: 63.86. - [Natural Language Explanation Generation on ECQA](): Natural Language Explanation Generation; ECQA; Human Evaluation Score: 41.92. - [Natural Language Explanation Generation on ComVE few-shot 60-shot](): Natural Language Explanation Generation; ComVE few-shot 60-shot; Accuracy: 63.71. - [Natural Language Explanation Generation on ECQA few-shot 60-shot](): Natural Language Explanation Generation; ECQA few-shot 60-shot; Accuracy: 11.14. - [Natural Language Explanation Generation on e-SNLI 60-shot](): Natural Language Explanation Generation; e-SNLI 60-shot; Accuracy: 34.91. - [Natural Language Explanation Generation on e-SNLI](): Natural Language Explanation Generation; E-SNLI; Human Evaluation Score: 29.63. - [Natural Language Explanation Generation on ComVE](): Natural Language Explanation Generation; ComVE; Human Evaluation Score: 40. - [Natural Language Explanation Generation on SBIC](): Natural Language Explanation Generation; SBIC; Human Evaluation Score: 54.44. ## Original sources and implementation [Original paper]() [PDF]()