Query-based meeting summarization on QMSum (test)
19.63ROUGE-2Oracle
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Oracle2023.05 | 19.63 | 43.8 | 39.1 | — | — | — | — | — | |
| SecEnc-W2022.09 | 13.4 | 37.8 | 33.4 | — | — | — | — | — | |
| SecEnc2022.09 | 13 | 37.1 | 32.6 | — | — | — | — | — | |
| Ranker-GeneratorExtractor Size(M)=1342023.05 | 12.23 | 35.51 | 31.28 | — | — | — | — | — | |
| Block-BARTadditional training data=true2022.09 | 12.1 | 36.6 | 32.4 | — | — | — | — | — | |
| BART-LS2022.09 | 12.1 | 37.9 | 33.1 | — | — | — | — | — | |
| RELREGExtractor Size(M)=13722023.05 | 11.91 | 34.91 | 30.73 | — | — | — | — | — | |
| Longformer2023.05 | 10.32 | 34.18 | 29.95 | — | — | — | — | — | |
| LED2022.09 | 10.3 | 34.2 | 30 | — | — | — | — | — | |
| RELREG-TTExtractor Size(M)=3292023.05 | 10.17 | 33.02 | 28.9 | — | — | — | — | — | |
| RankSUM(w/o re-ranking)Extractor Size(M)=1342023.05 | 9.73 | 33.02 | 29.15 | — | — | — | — | — | |
| DYLEExtractor Size(M)=5012023.05 | 9.71 | 34.42 | 30.1 | — | — | — | — | — | |
| DYLE2022.09 | 9.7 | 34.4 | 30.1 | — | — | — | — | — | |
| LEAD + BART2023.05 | 9.67 | 32.06 | 27.93 | — | — | — | — | — | |
| Simulator-GeneratorExtractor Size(M)=1342023.05 | 9.46 | 32.92 | 28.93 | — | — | — | — | — | |
| DialogLM2023.05 | 9.32 | 33.69 | 30.01 | — | — | — | — | — | |
| SUMMN2023.05 | 9.28 | 34.03 | 29.48 | — | — | — | — | — | |
| DialogLM2022.09 | 9.2 | 34 | 30 | — | — | — | — | — | |
| HMNet2023.05 | 8.67 | 32.29 | 28.17 | — | — | — | — | — | |
| Pointer Network + BARTExtractor Size(M)=4402023.05 | 8.53 | 31.74 | 28.21 | — | — | — | — | — | |
| Locator-GeneratorExtractor Size(M)=1342023.05 | 8.53 | 31.47 | 28.21 | — | — | — | — | — | |
| Pointer Network + PGNetExtractor Size(M)=4402023.05 | 8.47 | 31.37 | 27.08 | — | — | — | — | — | |
| BART-large2022.09 | 8 | 32.2 | 27.7 | — | — | — | — | — | |
| RaptorRAGBase LLM=Llama 3.1-8B2025.02 | 6.68 | — | — | 20.38 | 4.17 | 81.64 | 84.57 | 83.07 | |
| HippoRAG2Base LLM=Llama 3.1-8B2025.02 | 6.6 | — | — | 21.35 | 4.01 | 81.44 | 84.63 | 83 | |
| BART2023.05 | 6.37 | 29.2 | 25.49 | — | — | — | — | — | |
| IntegrationBase LLM=Llama 3.1-8B2025.02 | 6.34 | — | — | 21.97 | 3.8 | 80.89 | 84.47 | 82.63 | |
| RAGBase LLM=Llama 3.1-8B2025.02 | 6.32 | — | — | 21.5 | 3.8 | 81.03 | 84.45 | 82.69 | |
| PGNet2023.05 | 5.98 | 28.74 | 25.13 | — | — | — | — | — | |
| Community-GraphRAG (Local)Base LLM=Llama 3.1-8B2025.02 | 5.64 | — | — | 20.54 | 3.35 | 80.63 | 84.13 | 82.34 | |
| KG-GraphRAG (Triplets+Text)Base LLM=Llama 3.1-8B2025.02 | 5.38 | — | — | 16.83 | 3.32 | 80.92 | 83.64 | 82.25 | |
| KG-GraphRAG (Triplets only)Base LLM=Llama 3.1-8B2025.02 | 4.15 | — | — | 13.71 | 2.55 | 80.16 | 82.96 | 81.52 | |
| Community-GraphRAG (Global)Base LLM=Llama 3.1-8B2025.02 | 3.23 | — | — | 10.54 | 1.97 | 79.79 | 82.47 | 81.1 | |
| TextRank2023.05 | 2.69 | 16.27 | 15.41 | — | — | — | — | — |