Meeting Summarization on AMI manual transcriptions
42.43ROUGE-1 Recall (R)Our System (KeyRank)
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Our System (KeyRank)Summary word budget=350 and 450 (macro-averaged), Input transcription=ASR2018.05 | 42.43 | 35.01 | 37.86 | 8.72 | 7.29 | 7.84 | 16.19 | 13.76 | 14.71 | |
| Our SystemSummary size=350 and 450 words (macro-averaged), Transcription type=manual2018.05 | 42.03 | 34.77 | 37.53 | 8.87 | 7.56 | 8.06 | 15.92 | 14.08 | 14.76 | |
| Our System (FluCovRank)Summary word budget=350 and 450 (macro-averaged), Input transcription=ASR2018.05 | 41.84 | 34.61 | 37.37 | 8.29 | 6.92 | 7.45 | 16.28 | 13.48 | 14.58 | |
| Our SystemSummary word budget=350 and 450 (macro-averaged), Input transcription=ASR2018.05 | 41.83 | 34.44 | 37.25 | 8.22 | 6.95 | 7.43 | 15.83 | 13.7 | 14.51 | |
| Our System (FluCovRank)Summary size=350 and 450 words (macro-averaged), Transcription type=manual2018.05 | 41.73 | 34.5 | 37.27 | 8.45 | 7.05 | 7.6 | 16.08 | 13.47 | 14.49 | |
| Our System (Baseline)Summary word budget=350 and 450 (macro-averaged), Input transcription=ASR2018.05 | 41.56 | 34.37 | 37.11 | 7.88 | 6.66 | 7.11 | 15.36 | 13.2 | 14.02 | |
| CoreRank SubmodularSummary word budget=350 and 450 (macro-averaged), Input transcription=ASR2018.05 | 41.14 | 32.93 | 36.13 | 8.06 | 6.88 | 7.33 | 14.84 | 13.91 | 14.18 | |
| Our System (Baseline)Summary size=350 and 450 words (macro-averaged), Transcription type=manual2018.05 | 40.88 | 33.96 | 36.58 | 8.13 | 6.95 | 7.39 | 15.17 | 13.25 | 13.97 | |
| Our System (KeyRank)Summary size=350 and 450 words (macro-averaged), Transcription type=manual2018.05 | 40.87 | 33.91 | 36.56 | 8.42 | 7.12 | 7.62 | 15.5 | 13.48 | 14.25 | |
| PageRank SubmodularSummary word budget=350 and 450 (macro-averaged), Input transcription=ASR2018.05 | 40.84 | 33.08 | 36.1 | 8.27 | 6.88 | 7.42 | 15.37 | 13.71 | 14.32 | |
| PageRank SubmodularSummary size=350 and 450 words (macro-averaged), Transcription type=manual2018.05 | 40.58 | 32.87 | 35.86 | 9.2 | 7.77 | 8.32 | 15.59 | 14.14 | 14.64 | |
| OracleSummary size=350 and 450 words (macro-averaged), Transcription type=manual2018.05 | 40.49 | 34.65 | 36.73 | 8.07 | 7.35 | 7.55 | 15 | 14.03 | 14.26 | |
| OracleSummary word budget=350 and 450 (macro-averaged), Input transcription=Oracle (Human Extractive)2018.05 | 40.49 | 34.65 | 36.73 | 8.07 | 7.35 | 7.55 | 15 | 14.03 | 14.26 | |
| TextRankSummary word budget=350 and 450 (macro-averaged), Input transcription=ASR2018.05 | 39.55 | 32.6 | 35.25 | 7.67 | 6.43 | 6.9 | 14.87 | 12.87 | 13.62 | |
| TextRankSummary size=350 and 450 words (macro-averaged), Transcription type=manual2018.05 | 39.47 | 32.57 | 35.19 | 7.74 | 6.62 | 7.05 | 14.8 | 13.03 | 13.69 | |
| RandomSummary word budget=350 and 450 (macro-averaged), Input transcription=ASR2018.05 | 39.42 | 32.48 | 35.13 | 6.88 | 5.89 | 6.26 | 14.07 | 12.7 | 13.17 | |
| ClusterRankSummary word budget=350 and 450 (macro-averaged), Input transcription=ASR2018.05 | 39.36 | 32.53 | 35.14 | 7.14 | 6.05 | 6.46 | 14.34 | 12.8 | 13.35 | |
| RandomSummary size=350 and 450 words (macro-averaged), Transcription type=manual2018.05 | 39.29 | 32.38 | 35.01 | 7.14 | 6.16 | 6.52 | 14.16 | 12.95 | 13.35 | |
| CoreRank SubmodularSummary size=350 and 450 words (macro-averaged), Transcription type=manual2018.05 | 38.95 | 31.49 | 34.38 | 7.85 | 6.81 | 7.2 | 14.08 | 13.55 | 13.61 | |
| ClusterRankSummary size=350 and 450 words (macro-averaged), Transcription type=manual2018.05 | 38.32 | 31.51 | 34.1 | 6.93 | 5.95 | 6.31 | 13.69 | 12.4 | 12.84 | |
| Longest GreedySummary word budget=350 and 450 (macro-averaged), Input transcription=ASR2018.05 | 37.31 | 30.93 | 33.35 | 5.77 | 4.71 | 5.11 | 13.79 | 11.11 | 12.15 | |
| Longest GreedySummary size=350 and 450 words (macro-averaged), Transcription type=manual2018.05 | 36.73 | 30.39 | 32.78 | 5.52 | 4.58 | 4.93 | 13.52 | 10.91 | 11.93 |