Summarization on PubMed (test)
61.99ROUGE-1ORACLE
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| ORACLE2021.07 | 61.99 | 34.95 | 56.76 | |
| OracleModel Category=Baseline2022.11 | 61.99 | 34.95 | 56.76 | |
| ORACLESystem Category=Extractive2022.10 | 61.49 | 34.7 | 55.92 | |
| Top Down TransformerPooling Strategy=OracleAdaPool2022.03 | 55.15 | 26.55 | 50.25 | |
| ORACLEinput length=15k tok.2022.03 | 53.04 | 29.08 | 48.31 | |
| ORACLEinput length=9,600 tok.2022.03 | 52.8 | 28.95 | 48.08 | |
| Top-down (AdaP)# Param=660M*2022.09 | 51.1 | 23.3 | 46.5 | |
| Top Down TransformerPooling Strategy=AdaPool, Number of Parameters=464M2022.03 | 51.05 | 23.26 | 46.47 | |
| BART-LS# Param=440M2022.09 | 50.3 | 24.3 | 46.3 | |
| LongT5Model size=xl, Input length=16k, Attention mechanism=TGlobal2021.12 | 50.23 | 24.76 | 46.67 | |
| LongT5-xl# Param=3B2022.09 | 50.2 | 24.8 | 46.7 | |
| LongT5-large# Param=750M2022.09 | 50 | 24.7 | 46.5 | |
| LongT5Model size=large, Input length=16k, Attention mechanism=TGlobal2021.12 | 49.98 | 24.69 | 46.46 | |
| GoSumModel Category=Extractive2022.11 | 49.83 | 23.56 | 45.1 | |
| ORACLEinput length=4,096 tok.2022.03 | 49.73 | 27.29 | 45.26 | |
| Lodoss-full-LGSystem Category=Extractive, Loss Objective=Lsum + Lseg + bLDPP, Strategy=Local-Global context2022.10 | 49.38 | 23.89 | 44.84 | |
| MemSumMethod Type=Extractive2021.07 | 49.25 | 22.94 | 44.42 | |
| MemSumModel Category=Extractive2022.11 | 49.25 | 22.94 | 44.42 | |
| Lodoss-fullSystem Category=Extractive, Loss Objective=Lsum + Lseg + bLDPP2022.10 | 48.93 | 23.51 | 44.4 | |
| Topic-GraphSumModel Category=Extractive2022.11 | 48.85 | 21.76 | 35.19 | |
| Lodoss-jointSystem Category=Extractive, Loss Objective=Lsum + Lseg2022.10 | 48.83 | 23.13 | 44.23 | |
| HAT-BARTBackbone=BART2021.12 | 48.36 | 21.43 | 37 | |
| HATModel Category=Abstractive2022.11 | 48.36 | 21.43 | 37 | |
| HAT-BARTSource sequence length=30722021.04 | 48.36 | 21.43 | 37 | |
| Transformer-BARTSource sequence length=30722021.04 | 48.35 | 21.43 | 36.9 | |
| Top Down TransformerPooling Strategy=AvgPool, Number of Parameters=464M2022.03 | 48.34 | 21.4 | 44.22 | |
| Top-down (AvgP)# Param=460M2022.09 | 48.3 | 21.4 | 44.2 | |
| HATSystem Category=Abstractive2022.10 | 48.25 | 21.35 | 36.69 | |
| PageSum# Param=440M2022.09 | 48.2 | 21.1 | 44.3 | |
| HEROSModel Category=Extractive2022.11 | 48.14 | 21.82 | 43.33 | |
| LSH2022.03 | 48.12 | 21.06 | 42.72 | |
| Hepos-LSHMethod Type=Abstractive2021.07 | 48.12 | 21.06 | 42.72 | |
| Hepos-LSHModel Category=Abstractive2022.11 | 48.12 | 21.06 | 42.72 | |
| LSHSource sequence length=71682021.04 | 48.12 | 21.06 | 42.72 | |
| Lodoss-baseSystem Category=Extractive, Loss Objective=Lsum2022.10 | 48.1 | 22.53 | 43.51 | |
| BART-Hepos# Param=440M2022.09 | 48.1 | 21.1 | 42.7 | |
| LoBART(4k)+MCSType=Mix, Input sequence length=4k, ORCpad-rand=true2021.05 | 48.06 | 20.96 | 43.56 | |
| Hepos-SinkhornMethod Type=Abstractive2021.07 | 47.93 | 20.74 | 42.58 | |
| Hepos-SinkhornModel Category=Abstractive2022.11 | 47.93 | 20.74 | 42.58 | |
| LoBART(4k)Type=Abs, Input sequence length=4k, ORCpad-rand=true2021.05 | 47.47 | 20.47 | 43.02 | |
| NeuSumMethod Type=Extractive2021.07 | 47.46 | 21.92 | 42.87 | |
| NeuSumModel Category=Extractive2022.11 | 47.46 | 21.92 | 42.87 | |
| LED# Param=460M2022.09 | 47 | 20.2 | 42.9 | |
| LongformerNumber of Parameters=460M2022.03 | 46.97 | 20.23 | 42.88 | |
| SSN-DM2022.03 | 46.73 | 21 | 34.1 | |
| HEROS w/o content rankingModel Category=Extractive2022.11 | 46.63 | 20.63 | 42.01 | |
| HiStruct+Backbone=Longformer-base, input length=15k tok., variant=SHE+STE(classified)2022.03 | 46.59 | 20.39 | 42.11 | |
| BART(1k)+MCSType=Mix, Input sequence length=1k, TRC=true2021.05 | 46.49 | 19.45 | 42.04 | |
| HiStruct+Backbone=Longformer-base, input length=15k tok., variant=SHE+STE2022.03 | 46.49 | 20.29 | 42.02 | |
| HiStruct+Backbone=Longformer-large, input length=15k tok., variant=SHE+STE(classified)2022.03 | 46.38 | 20.17 | 41.92 | |
| DANCERType=Abs2021.05 | 46.34 | 19.97 | 42.42 | |
| Dancer2022.03 | 46.34 | 19.97 | 42.42 | |
| DancerMethod Type=Abstractive2021.07 | 46.34 | 19.97 | 42.42 | |
| DANCER PEGASUSApproach=Abstractive, Year=20202022.03 | 46.34 | 19.97 | 42.42 | |
| DANCER PEGASUSBackbone=PEGASUS2021.12 | 46.34 | 19.97 | 42.42 | |
| DancerModel Category=Abstractive2022.11 | 46.34 | 19.97 | 42.42 | |
| BigBird(3k)Type=Abs, Input sequence length=3k2021.05 | 46.32 | 20.65 | 42.33 | |
| BigBirdNumber of Parameters=577M2022.03 | 46.32 | 20.65 | 42.33 | |
| BigBirdMethod Type=Abstractive2021.07 | 46.32 | 20.65 | 42.33 | |
| BigBird PEGASUSApproach=Abstractive, Year=20202022.03 | 46.32 | 20.65 | 42.33 | |
| BigBird-PEGASUSModel size=large, Backbone=PEGASUS2021.12 | 46.32 | 20.65 | 42.33 | |
| BigBird-largeSystem Category=Abstractive, Model Scale=large2022.10 | 46.32 | 20.65 | 42.33 | |
| BigBird-largeModel Category=Abstractive2022.11 | 46.32 | 20.65 | 42.33 | |
| BigBirdSource sequence length=40962021.04 | 46.32 | 20.65 | 42.33 | |
| BigBird# Param=580M2022.09 | 46.3 | 20.7 | 42.3 | |
| Our ModelMethod description=Flow-based Neural Topic Model2021.09 | 45.99 | 20.49 | 41.25 | |
| PegasusType=Abs2021.05 | 45.97 | 20.15 | 41.34 | |
| PEGASUSMethod Type=Abstractive2021.07 | 45.97 | 20.15 | 41.34 | |
| PEGASUSModel Category=Abstractive2022.11 | 45.97 | 20.15 | 41.34 | |
| PEGASUSSource sequence length=40962021.04 | 45.97 | 20.15 | 41.34 | |
| HiStruct+Backbone=Longformer-base, input length=15k tok., variant=SHE2022.03 | 45.76 | 19.64 | 41.34 | |
| HiStruct+Backbone=Longformer-large, input length=15k tok., variant=SHE2022.03 | 45.67 | 19.6 | 41.26 | |
| PEGASUSApproach=Abstractive, Year=20202022.03 | 45.49 | 19.9 | 42.42 | |
| MMR-Select+Approach=Extractive2022.03 | 45.39 | 20.37 | 40.99 | |
| ExtSum-LG + RdLossSystem Category=Extractive, Loss Objective=Redundancy Loss2022.10 | 45.39 | 20.37 | 40.99 | |
| ExtSum-LG+RdType=Ext2021.05 | 45.3 | 20.42 | 40.95 | |
| RLossApproach=Extractive2022.03 | 45.3 | 20.42 | 40.95 | |
| Local-Global + RdLossModel Category=Extractive2022.11 | 45.3 | 20.42 | 40.95 | |
| ExtSum-LG+RdLoss2021.09 | 45.3 | 20.42 | 40.95 | |
| Local-GlobalModel Category=Extractive2022.11 | 45.18 | 20.2 | 40.72 | |
| BART(1k)Type=Abs, Input sequence length=1k2021.05 | 45.06 | 18.27 | 40.84 | |
| Sent-CLFMethod Type=Extractive2021.07 | 45.01 | 19.91 | 41.16 | |
| Sent-CLFApproach=Extractive, Year=20202022.03 | 45.01 | 19.91 | 41.16 | |
| Sent-PTRSystem Category=Extractive2022.10 | 45.01 | 19.91 | 41.16 | |
| Sent-CLFModel Category=Extractive2022.11 | 45.01 | 19.91 | 41.16 | |
| Atten-ContMethod Type=Extractive2021.07 | 44.85 | 19.7 | 31.43 | |
| ExtSum-LGSystem Category=Extractive2022.10 | 44.85 | 19.7 | 31.43 | |
| PEGASUS2021.09 | 44.29 | 19.19 | 40.42 | |
| PegasusNumber of Parameters=568M2022.03 | 44.21 | 16.95 | 38.83 | |
| SummaRuNNerMethod Type=Extractive2021.07 | 43.89 | 18.78 | 30.36 | |
| BigBird-baseSystem Category=Abstractive, Model Scale=base2022.10 | 43.7 | 19.32 | 39.99 | |
| BigBird-baseModel Category=Abstractive2022.11 | 43.7 | 19.32 | 39.99 | |
| Sent-PTRMethod Type=Extractive2021.07 | 43.3 | 17.92 | 39.47 | |
| Sent-PTRApproach=Extractive, Year=20202022.03 | 43.3 | 17.92 | 39.47 | |
| Sent-PTRModel Category=Extractive2022.11 | 43.3 | 17.92 | 39.47 | |
| Ext+TLMType=Mix2021.05 | 42.13 | 16.27 | 39.21 | |
| TLM-I+E2022.03 | 42.13 | 16.27 | 39.21 | |
| TLM-I+E(G,M)Approach=Hybrid, Year=20202022.03 | 42.13 | 16.27 | 39.21 | |
| TLM-I+ESystem Category=Abstractive2022.10 | 42.13 | 16.27 | 39.21 | |
| Sent-CLF2021.09 | 42.13 | 16.27 | 39.21 |