AMR-to-text generation on LDC2017T10 (test)
49.72BLEUT5large + STA (2M)
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| T5large + STA (2M)task-adaptive pretraining=STA, pretraining data volume=2M2020.07 | 49.72 | 45.43 | — | — | — | — | — | 64.24 | |
| T5large + STA (200K)task-adaptive pretraining=STA, pretraining data volume=200K2020.07 | 48.02 | 44.85 | — | — | — | — | — | 63.86 | |
| BARTlarge + STA (2M)task-adaptive pretraining=STA, pretraining data volume=2M2020.07 | 47.51 | 44.7 | — | — | — | — | — | 62.27 | |
| STRUCTADAPT-RGCNModel Scale=T5-large, Fine-tuning Strategy=StructAdapt, Graph Convolution=RGCN, Percentage of Trainable Parameters=5.1%2021.03 | 46.6 | — | — | — | 72.9 | 79.6 | 96.3 | — | |
| T5large + LMAtask-adaptive pretraining=LMA2020.07 | 46.06 | 44.05 | — | — | — | — | — | 62.59 | |
| Ribeiro et al.2021.03 | 45.8 | — | — | — | 72.5 | — | — | — | |
| STRUCTADAPT-GCNModel Scale=T5-large, Fine-tuning Strategy=StructAdapt, Graph Convolution=GCN, Percentage of Trainable Parameters=6.8%2021.03 | 45.8 | — | — | — | 72.5 | 79.3 | 96.2 | — | |
| T5largePLM-based=true2020.07 | 45.8 | 43.85 | — | — | — | — | — | 61.93 | |
| Hoyle et al.2021.03 | 44.9 | — | — | — | — | 76.54 | — | — | |
| BARTlarge + STA (200K)task-adaptive pretraining=STA, pretraining data volume=200K2020.07 | 44.72 | 43.65 | — | — | — | — | — | 61.03 | |
| STRUCTADAPT-GCNModel Scale=T5-large, Fine-tuning Strategy=StructAdapt, Graph Convolution=GCN, Percentage of Trainable Parameters=1.7%2021.03 | 44.1 | — | — | — | 71.8 | 79.1 | 96.1 | — | |
| STRUCTADAPT-RGCNModel Scale=T5-base, Fine-tuning Strategy=StructAdapt, Graph Convolution=RGCN, Percentage of Trainable Parameters=6.3%2021.03 | 44 | — | — | — | 71.2 | 79.4 | 95.9 | — | |
| BARTlarge + LMAtask-adaptive pretraining=LMA2020.07 | 43.94 | 42.36 | — | — | — | — | — | 58.54 | |
| BARTlargePLM-based=true2020.07 | 43.47 | 42.88 | — | — | — | — | — | 60.42 | |
| ADAPTModel Scale=T5-large, Fine-tuning Strategy=Adapter, Percentage of Trainable Parameters=6.8%2021.03 | 42.9 | — | — | — | 71.6 | 78.9 | 96.1 | — | |
| T5basePLM-based=true2020.07 | 42.54 | 42.62 | — | — | — | — | — | 60.59 | |
| FINE-TUNEModel Scale=T5-large, Fine-tuning Strategy=Full fine-tuning2021.03 | 41.2 | — | — | — | 70.2 | 78 | 95.8 | — | |
| STRUCTADAPT-GCNModel Scale=T5-base, Fine-tuning Strategy=StructAdapt, Graph Convolution=GCN, Percentage of Trainable Parameters=8.5%2021.03 | 41 | — | — | — | 70 | 78.4 | 95.7 | — | |
| STRUCTADAPT-GCNModel Scale=T5-base, Fine-tuning Strategy=StructAdapt, Graph Convolution=GCN, Percentage of Trainable Parameters=2.1%2021.03 | 39 | — | — | — | 69.1 | 78.4 | 95.7 | — | |
| ADAPTModel Scale=T5-base, Fine-tuning Strategy=Adapter, Percentage of Trainable Parameters=8.5%2021.03 | 38.7 | — | — | — | 69.2 | 78.3 | 95.6 | — | |
| T5smallPLM-based=true2020.07 | 38.45 | 40.86 | — | — | — | — | — | 57.95 | |
| FINE-TUNEModel Scale=T5-base, Fine-tuning Strategy=Full fine-tuning2021.03 | 38.3 | — | — | — | 68.6 | 77.8 | 95.5 | — | |
| DATATUNER_FCSFC classifier=true2020.04 | 37.7 | 38.9 | 65.1 | 3.9 | — | — | — | — | |
| Harkous et al.2021.03 | 37.7 | — | — | — | — | — | — | — | |
| Harkous et al. (2020)PLM-based=true2020.07 | 37.7 | 38.9 | — | — | — | — | — | — | |
| FT-BOTTOM2Model Scale=T5-large, Fine-tuning Strategy=Bottom 2 layers, Percentage of Trainable Parameters=7.9%2021.03 | 37.6 | — | — | — | 68 | 77.2 | 95.5 | — | |
| DATATUNER_NO_FCSFC classifier=false2020.04 | 37.2 | 38.4 | 65 | 3.9 | — | — | — | — | |
| BARTbasePLM-based=true2020.07 | 36.71 | 38.64 | — | — | — | — | — | 52.47 | |
| FT-BOTTOM2Model Scale=T5-base, Fine-tuning Strategy=Bottom 2 layers, Percentage of Trainable Parameters=14.8%2021.03 | 35.9 | — | — | — | 67 | 76.9 | 95.3 | — | |
| DATATUNER_NO_FC/FSSFC classifier=false, variant=FS2020.04 | 35.6 | 37.3 | 64.4 | 3.8 | — | — | — | — | |
| RA-Trans-F-oursAuxiliary training losses=Both2021.02 | 34.21 | 38 | — | — | — | — | — | — | |
| RA-Trans-F-oursAuxiliary training losses=Loss 22021.02 | 34.13 | 37.8 | — | — | — | — | — | — | |
| Yao et al. (2020)2020.07 | 34.1 | 38.1 | — | — | — | — | — | — | |
| RA-Trans-F-oursAuxiliary training losses=Loss 12021.02 | 33.98 | 37.5 | — | — | — | — | — | — | |
| Wang et al. (2020)2020.07 | 33.9 | 37.1 | — | — | — | — | — | — | |
| Zhang et al.2021.03 | 33.6 | — | — | — | 63.2 | — | — | — | |
| GPT-2Lre-scoring=true2021.02 | 33.02 | 37.68 | — | — | — | — | — | — | |
| Mager et al. (2020)PLM-based=true2020.07 | 33.02 | 37.68 | — | — | — | — | — | — | |
| Mager et al.2021.03 | 33 | — | — | — | 63.9 | — | — | — | |
| GPT-2Lre-scoring=false2021.02 | 32.47 | 36.8 | — | — | — | — | — | — | |
| Zhao et al. (2020b)2020.07 | 32.46 | 36.78 | — | — | — | — | — | — | |
| RA-Trans-CNNModel architecture=Convolutional Neural Network2021.02 | 31.82 | 36.38 | — | — | — | — | — | — | |
| Zhu et al. (2019)2020.07 | 31.82 | 36.38 | — | — | — | — | — | — | |
| Zhu et al. (2019)2020.04 | 31.8 | 36.4 | — | — | — | — | — | — | |
| RA-Trans-F-oursAuxiliary training losses=none2021.02 | 31.77 | 37.2 | — | — | — | — | — | — | |
| Guo et al. (2019)2020.04 | 30.4 | — | — | — | — | — | — | — | |
| FT-TOP2Model Scale=T5-base, Fine-tuning Strategy=Top 2 layers, Percentage of Trainable Parameters=14.8%2021.03 | 29.9 | — | — | — | 63 | 74.1 | 94.4 | — | |
| FT-TOP2Model Scale=T5-large, Fine-tuning Strategy=Top 2 layers, Percentage of Trainable Parameters=7.9%2021.03 | 28.8 | — | — | — | 61.8 | 73.9 | 94.1 | — | |
| Ribeiro et al. (2019)2020.04 | 27.9 | 33.2 | — | — | — | — | — | — | |
| Ribeiro et al. (2019)2020.07 | 27.87 | 33.21 | — | — | — | — | — | — | |
| DCGCN2021.02 | 27.6 | — | — | — | — | — | — | — | |
| GRAPHEncoder=Graph-based GCNSEQ, Modeling reentrancies=true2019.03 | 24.54 | 24.07 | — | — | — | — | — | — | |
| TREEEncoder=Tree-based GCNSEQ, Modeling reentrancies=false2019.03 | 24.06 | 23.62 | — | — | — | — | — | — | |
| Beck et al. (2018)Encoder=Graph-based2019.03 | 23.3 | — | — | — | — | — | — | — | |
| SEQEncoder=Sequential2019.03 | 22.19 | 22.68 | — | — | — | — | — | — |