KP-Comment Matching on AMAZONKP (test)
76.2PrecisionPAKPA
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| PAKPABase Framework=Frozen Retriever + KPA, LLM/Model Variant=PAKPA2025.06 | 76.2 | 52 | 61.9 | 6.68 | |
| Frozen Retriever + prompt LLMBase Framework=Frozen Retriever + prompt LLM, LLM/Model Variant=GPT-4-Turbo2025.06 | 74.6 | 20 | 31.3 | 16.63 | |
| QQSUM-RAGBase Framework=QQSUM-RAG, LLM/Model Variant=Mistral2025.06 | 69.4 | 86.9 | 79.2 | 4.24 | |
| (Retriever + LLM)co-trainBase Framework=(Retriever + LLM)co-train, LLM/Model Variant=Mistral2025.06 | 56.7 | 24.9 | 34.6 | 18.1 | |
| QQSUM-RAGBase Framework=QQSUM-RAG, LLM/Model Variant=Vicuna2025.06 | 53.8 | 68.4 | 60.2 | 7.83 | |
| Frozen Retriever + prompt LLMBase Framework=Frozen Retriever + prompt LLM, LLM/Model Variant=Mistral2025.06 | 49.8 | 21.4 | 30 | 19.14 | |
| (Retriever + LLM)co-trainBase Framework=(Retriever + LLM)co-train, LLM/Model Variant=Vicuna2025.06 | 44.2 | 9.4 | 15.4 | 30.13 | |
| Frozen Retriever + prompt LLMBase Framework=Frozen Retriever + prompt LLM, LLM/Model Variant=Vicuna2025.06 | 43.9 | 18.5 | 26 | 21.52 | |
| RKPA-BaseBase Framework=Frozen Retriever + KPA, LLM/Model Variant=RKPA-Base2025.06 | 37.1 | 31.4 | 34 | 15.62 |