Meeting Summarization on ICSI manual transcriptions
0.4003ROUGE-1 RecallOur System (Baseline)
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Our System (Baseline)Summary size=350 and 450 words (macro-averaged), Transcription type=manual2018.05 | 0.4003 | 0.302 | 0.3411 | 0.0665 | 0.0551 | 0.0598 | 0.1465 | 0.1137 | 0.127 | — | — | — | |
| Our System (KeyRank)Summary size=350 and 450 words (macro-averaged), Transcription type=manual2018.05 | 0.3955 | 0.2979 | 0.3368 | 0.0632 | 0.0519 | 0.0564 | 0.1463 | 0.1099 | 0.1247 | — | — | — | |
| Our SystemSummary size=350 and 450 words (macro-averaged), Transcription type=manual2018.05 | 0.3857 | 0.293 | 0.3293 | 0.058 | 0.0474 | 0.0514 | 0.1392 | 0.1079 | 0.1204 | — | — | — | |
| Our System (FluCovRank)Summary size=350 and 450 words (macro-averaged), Transcription type=manual2018.05 | 0.3857 | 0.2921 | 0.3295 | 0.0638 | 0.0508 | 0.056 | 0.1438 | 0.1062 | 0.1213 | — | — | — | |
| OracleSummary size=350 and 450 words (macro-averaged), Transcription type=manual2018.05 | 0.3791 | 0.2839 | 0.3212 | 0.0573 | 0.0482 | 0.0518 | 0.1335 | 0.1073 | 0.118 | — | — | — | |
| OracleSummary word budget=350 and 450 (macro-averaged), Input transcription=Oracle (Human Extractive)2018.05 | 0.3791 | 0.2839 | 0.3212 | 0.0573 | 0.0482 | 0.0518 | 0.1335 | 0.1073 | 0.118 | — | — | — | |
| PageRank SubmodularSummary size=350 and 450 words (macro-averaged), Transcription type=manual2018.05 | 0.3772 | 0.2886 | 0.3235 | 0.0635 | 0.0546 | 0.0582 | 0.1335 | 0.116 | 0.123 | — | — | — | |
| TextRankSummary size=350 and 450 words (macro-averaged), Transcription type=manual2018.05 | 0.376 | 0.2879 | 0.3232 | 0.0663 | 0.0553 | 0.0598 | 0.1418 | 0.1118 | 0.1241 | — | — | — | |
| RandomSummary size=350 and 450 words (macro-averaged), Transcription type=manual2018.05 | 0.3748 | 0.281 | 0.318 | 0.0541 | 0.0465 | 0.0495 | 0.1297 | 0.1067 | 0.1161 | — | — | — | |
| CoreRank SubmodularSummary size=350 and 450 words (macro-averaged), Transcription type=manual2018.05 | 0.3731 | 0.2951 | 0.3245 | 0.0559 | 0.0505 | 0.0524 | 0.1319 | 0.1108 | 0.1187 | — | — | — | |
| Longest GreedySummary size=350 and 450 words (macro-averaged), Transcription type=manual2018.05 | 0.3715 | 0.2821 | 0.3176 | 0.055 | 0.046 | 0.0498 | 0.1359 | 0.1003 | 0.1146 | — | — | — | |
| Our SystemSummary word budget=350 and 450 (macro-averaged), Input transcription=ASR2018.05 | 0.3699 | 0.2812 | 0.316 | 0.0541 | 0.0439 | 0.0479 | 0.131 | 0.1017 | 0.1135 | — | — | — | |
| Our System (Baseline)Summary word budget=350 and 450 (macro-averaged), Input transcription=ASR2018.05 | 0.3639 | 0.272 | 0.308 | 0.0519 | 0.0412 | 0.0455 | 0.1259 | 0.097 | 0.1086 | — | — | — | |
| Our System (FluCovRank)Summary word budget=350 and 450 (macro-averaged), Input transcription=ASR2018.05 | 0.3627 | 0.2756 | 0.31 | 0.0556 | 0.0435 | 0.0483 | 0.1347 | 0.0985 | 0.1129 | — | — | — | |
| PageRank SubmodularSummary word budget=350 and 450 (macro-averaged), Input transcription=ASR2018.05 | 0.3605 | 0.2669 | 0.304 | 0.0482 | 0.0416 | 0.0442 | 0.1219 | 0.1039 | 0.1114 | — | — | — | |
| Our System (KeyRank)Summary word budget=350 and 450 (macro-averaged), Input transcription=ASR2018.05 | 0.3595 | 0.27 | 0.3052 | 0.0464 | 0.0364 | 0.0404 | 0.1243 | 0.0923 | 0.105 | — | — | — | |
| ClusterRankSummary size=350 and 450 words (macro-averaged), Transcription type=manual2018.05 | 0.3566 | 0.2658 | 0.3014 | 0.0453 | 0.0399 | 0.0421 | 0.121 | 0.0971 | 0.1069 | — | — | — | |
| Longest GreedySummary word budget=350 and 450 (macro-averaged), Input transcription=ASR2018.05 | 0.3557 | 0.2674 | 0.3023 | 0.0484 | 0.0388 | 0.0427 | 0.1309 | 0.0946 | 0.109 | — | — | — | |
| CoreRank SubmodularSummary word budget=350 and 450 (macro-averaged), Input transcription=ASR2018.05 | 0.3522 | 0.2634 | 0.2982 | 0.0436 | 0.0376 | 0.04 | 0.1211 | 0.0958 | 0.1061 | — | — | — | |
| TextRankSummary word budget=350 and 450 (macro-averaged), Input transcription=ASR2018.05 | 0.3489 | 0.2633 | 0.297 | 0.046 | 0.0374 | 0.0409 | 0.1242 | 0.0943 | 0.1064 | — | — | — | |
| RandomSummary word budget=350 and 450 (macro-averaged), Input transcription=ASR2018.05 | 0.3478 | 0.2575 | 0.2928 | 0.0419 | 0.0351 | 0.0378 | 0.1161 | 0.0937 | 0.1029 | — | — | — | |
| ClusterRankSummary word budget=350 and 450 (macro-averaged), Input transcription=ASR2018.05 | 0.3263 | 0.2444 | 0.2764 | 0.0403 | 0.0344 | 0.0368 | 0.1104 | 0.0888 | 0.0977 | — | — | — | |
| HMNet2021.06 | — | — | — | — | — | — | — | — | — | 46.28 | 10.6 | — | |
| Longformer-BART2021.06 | — | — | — | — | — | — | — | — | — | 43.03 | 12.14 | 40.26 | |
| Longformer-BART-argvariation=arg-filtered2021.06 | — | — | — | — | — | — | — | — | — | 44.17 | 11.69 | 41.33 |