Key Point Summary Textual Quality on AMAZONKP (test)
0.256ROUGE-1QQSUM-RAG
Evaluation Results
| Method | Links | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| QQSUM-RAGLLM Backbone=Mistral2025.06 | 0.256 | 0.061 | 0.22 | 0.39 | 0.29 | 0.33 | 0.37 | 0.51 | 0.41 | 0.46 | 0.49 | 0.88 | 0.82 | 0.85 | 0.36 | |
| QQSUM-RAGLLM Backbone=Vicuna2025.06 | 0.222 | 0.078 | 0.204 | 0.38 | 0.26 | 0.31 | 0.53 | 0.49 | 0.39 | 0.44 | 0.54 | 0.87 | 0.81 | 0.84 | 0.36 | |
| Frozen Retriever + prompt LLMLLM Backbone=Mistral2025.06 | 0.21 | 0.055 | 0.191 | 0.33 | 0.26 | 0.29 | 0.51 | 0.46 | 0.38 | 0.42 | 0.55 | 0.79 | 0.8 | 0.79 | 0.41 | |
| Retriever + LLM (co-train)LLM Backbone=Mistral2025.06 | 0.209 | 0.057 | 0.194 | 0.37 | 0.28 | 0.32 | 0.43 | 0.49 | 0.4 | 0.44 | 0.55 | 0.81 | 0.82 | 0.81 | 0.41 | |
| Frozen Retriever + prompt LLMLLM Backbone=GPT-4-Turbo2025.06 | 0.197 | 0.048 | 0.174 | 0.32 | 0.25 | 0.28 | 0.44 | 0.45 | 0.38 | 0.41 | 0.54 | 0.77 | 0.77 | 0.77 | 0.38 | |
| Frozen Retriever + KPAKPA Model Variant=PAKPA2025.06 | 0.179 | 0.027 | 0.162 | 0.34 | 0.28 | 0.31 | 0.46 | 0.47 | 0.41 | 0.44 | 0.54 | 0.79 | 0.8 | 0.8 | 0.36 | |
| Retriever + LLM (co-train)LLM Backbone=Vicuna2025.06 | 0.174 | 0.041 | 0.161 | 0.37 | 0.26 | 0.31 | 0.48 | 0.48 | 0.38 | 0.42 | 0.58 | 0.78 | 0.78 | 0.78 | 0.41 | |
| Frozen Retriever + prompt LLMLLM Backbone=Vicuna2025.06 | 0.164 | 0.059 | 0.154 | 0.22 | 0.2 | 0.21 | 0.48 | 0.46 | 0.31 | 0.37 | 0.59 | 0.73 | 0.73 | 0.73 | 0.41 | |
| Frozen Retriever + KPAKPA Model Variant=RKPA-Base2025.06 | 0.121 | 0.016 | 0.106 | 0.16 | 0.14 | 0.14 | 0.5 | 0.43 | 0.36 | 0.39 | 0.61 | 0.69 | 0.7 | 0.69 | 0.51 |