# SPARSEFIT: Few-shot Prompting with Sparse Fine-tuning for Jointly Generating Predictions and Natural Language Explanations Source: [SPARSEFIT: Few-shot Prompting with Sparse Fine-tuning for Jointly Generating Predictions and Natural Language Explanations]() Collection index: [llms.txt]() Data updated: 2026-01-04T04:32:49.069Z Authors: Jesus Solano, Mardhiyah Sanni, Oana-Maria Camburu, Pasquale Minervini Published: 2023-05-22T17:06:41.000Z ## Abstract Models that generate natural language explanations (NLEs) for their predictions have recently gained increasing interest. However, this approach usually demands large datasets of human-written NLEs for the ground-truth answers at training time, which can be expensive and potentially infeasible for some applications. When only a few NLEs are available (a few-shot setup), fine-tuning pre-trained language models (PLMs) in conjunction with prompt-based learning has recently shown promising results. However, PLMs typically have billions of parameters, making full fine-tuning expensive. We propose SparseFit, a sparse few-shot fine-tuning strategy that leverages discrete prompts to jointly generate predictions and NLEs. We experiment with SparseFit on three sizes of the T5 language model and four datasets and compare it against existing state-of-the-art Parameter-Efficient Fine-Tuning (PEFT) techniques. We find that fine-tuning only 6.8% of the model parameters leads to competitive results for both the task performance and the quality of the generated NLEs compared to full fine-tuning of the model and produces better results on average than other PEFT methods in terms of predictive accuracy and NLE quality. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Natural Language Explanation Generation on ECQA](): Natural Language Explanation Generation; ECQA; Human Evaluation Score: 73.33. - [Natural Language Explanation Generation on ComVE](): Natural Language Explanation Generation; ComVE; Human Evaluation Score: 70. - [Natural Language Explanation Generation on SBIC (test)](): Natural Language Explanation Generation; SBIC (test); Accuracy: 66.99. - [Natural Language Explanation Generation on ComVE few-shot 60-shot](): Natural Language Explanation Generation; ComVE few-shot 60-shot; Accuracy: 68.03. - [Natural Language Explanation Generation on ECQA few-shot 60-shot](): Natural Language Explanation Generation; ECQA few-shot 60-shot; Accuracy: 24.53. - [Natural Language Explanation Generation on e-SNLI 60-shot](): Natural Language Explanation Generation; e-SNLI 60-shot; Accuracy: 40.1. - [Natural Language Explanation Generation on e-SNLI](): Natural Language Explanation Generation; E-SNLI; Human Evaluation Score: 41.11. - [Natural Language Explanation Generation on SBIC](): Natural Language Explanation Generation; SBIC; Human Evaluation Score: 68.89. - [Natural Language Explanation Generation on SBIC 60-shot](): Natural Language Explanation Generation; SBIC 60-shot; Accuracy: 57.9. - [Natural Language Explanation Generation on ECQA (test)](): Natural Language Explanation Generation; ECQA (test); Accuracy: 55.81. - [Commonsense Validation and Explanation on ComVE (test)](): Commonsense Validation and Explanation; ComVE (test); Accuracy: 74.86. - [Natural Language Explanation Generation on e-SNLI (test)](): Natural Language Explanation Generation; e-SNLI (test); Accuracy: 82.62. ## Original sources and implementation [Original paper]() [PDF]()