# Language Generation on Vicuna (test) Source: [Language Generation on Vicuna (test)]() Collection index: [llms.txt]() Data updated: 2026-03-05T08:49:25.662Z Evaluation of language generation capabilities on the Vicuna test set, assessed using ROUGE-L scores. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Language Generation | | Dataset (including any stated split) | Vicuna (test) | | Metric | ROUGE-L | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 14 of 14 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [ROUGE-L](): Higher is better; unit: PERCENT. ## Recorded results | Method | ROUGE-L | Result date | Report paper | | --- | --- | --- | --- | | Teacher | 19.4 | Not recorded | 69a90e112d603e6de3e4083f | | VQAE | 18.7 | Not recorded | 69a90e112d603e6de3e4083f | | KD | 18.4 | Not recorded | 69a90e112d603e6de3e4083f | | SeqKD | 18.1 | Not recorded | 69a90e112d603e6de3e4083f | | SFT w/o KD | 17.5 | Not recorded | 69a90e112d603e6de3e4083f | | KD | 16.9 | Not recorded | 69a90e112d603e6de3e4083f | | Teacher | 16.3 | Not recorded | 69a90e112d603e6de3e4083f | | VQAE | 16.3 | Not recorded | 69a90e112d603e6de3e4083f | | SFT w/o KD | 16.1 | Not recorded | 69a90e112d603e6de3e4083f | | SeqKD | 15.9 | Not recorded | 69a90e112d603e6de3e4083f | | VQAE | 15.2 | Not recorded | 69a90e112d603e6de3e4083f | | SFT w/o KD | 14.7 | Not recorded | 69a90e112d603e6de3e4083f | | SeqKD | 14.3 | Not recorded | 69a90e112d603e6de3e4083f | | KD | 13.4 | Not recorded | 69a90e112d603e6de3e4083f | ### Teacher ROUGE-L: 19.4. Result ID: 69a943826dbb7075b952d125. [Reporting paper]() Recorded evaluation configuration: {"Model":"LLaMA","#Params":"13B"}. This record does not assert independent reproduction. ### VQAE ROUGE-L: 18.7. Result ID: 69a943826dbb7075b952d129. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA","#Params":"7B"}. This record does not assert independent reproduction. ### KD ROUGE-L: 18.4. Result ID: 69a943826dbb7075b952d127. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"LLaMA","#Params":"7B"}. This record does not assert independent reproduction. ### SeqKD ROUGE-L: 18.1. Result ID: 69a943826dbb7075b952d128. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"LLaMA","#Params":"7B"}. This record does not assert independent reproduction. ### SFT w/o KD ROUGE-L: 17.5. Result ID: 69a943826dbb7075b952d126. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"LLaMA","#Params":"7B"}. This record does not assert independent reproduction. ### KD ROUGE-L: 16.9. Result ID: 69a943826dbb7075b952d122. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"GPT-2","#Params":"760M"}. This record does not assert independent reproduction. ### Teacher ROUGE-L: 16.3. Result ID: 69a943826dbb7075b952d11c. [Reporting paper]() Recorded evaluation configuration: {"Model":"GPT-2","#Params":"1.5B"}. This record does not assert independent reproduction. ### VQAE ROUGE-L: 16.3. Result ID: 69a943826dbb7075b952d124. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-2","#Params":"760M"}. This record does not assert independent reproduction. ### SFT w/o KD ROUGE-L: 16.1. Result ID: 69a943826dbb7075b952d121. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"GPT-2","#Params":"760M"}. This record does not assert independent reproduction. ### SeqKD ROUGE-L: 15.9. Result ID: 69a943826dbb7075b952d123. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"GPT-2","#Params":"760M"}. This record does not assert independent reproduction. ### VQAE ROUGE-L: 15.2. Result ID: 69a943826dbb7075b952d120. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-2","#Params":"120M"}. This record does not assert independent reproduction. ### SFT w/o KD ROUGE-L: 14.7. Result ID: 69a943826dbb7075b952d11d. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"GPT-2","#Params":"120M"}. This record does not assert independent reproduction. ### SeqKD ROUGE-L: 14.3. Result ID: 69a943826dbb7075b952d11f. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"GPT-2","#Params":"120M"}. This record does not assert independent reproduction. ### KD ROUGE-L: 13.4. Result ID: 69a943826dbb7075b952d11e. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"GPT-2","#Params":"120M"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.