Abstractive Summarization on QMSum
6.75BLEUChatGPT
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| ChatGPT2023.11 | 6.75 | 23.36 | 7.25 | |
| Orca-2-13BSystem Message=Direct2023.11 | 6.62 | 23.54 | 7.44 | |
| Orca-2-7BSystem Message=Direct2023.11 | 6.54 | 23.13 | 7.44 | |
| Orca-1-13B2023.11 | 6.36 | 23.09 | 7.43 | |
| WizardLM-70B2023.11 | 5.8 | 21.87 | 7.6 | |
| GPT-42023.11 | 5.79 | 22.46 | 6.98 | |
| LLAMA-2-Chat-70B2023.11 | 5.73 | 21.82 | 7.51 | |
| WizardLM-13B2023.11 | 5.08 | 20.64 | 7.5 | |
| LLAMA-2-Chat-13B2023.11 | 4.58 | 19.77 | 6.9 | |
| Orca-2-7BSystem Message=Cautious2023.11 | 3.49 | 19.45 | 9.11 | |
| Orca-2-13BSystem Message=Cautious2023.11 | 3.41 | 19.43 | 8.75 |