Data-to-text generation on WebNLG (test)
64.11BLEUKGPT-Seq
Evaluation results
| Method | Links | |||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| KGPT-SeqEncoder architecture=Seq, Pre-training=true2020.10 | 64.11 | 46.3 | — | 74.57 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| KGPT-GraphEncoder architecture=Graph, Pre-training=true2020.10 | 63.84 | 46.1 | — | 74.04 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| KGPT-GraphEncoder architecture=Graph, Pre-training=false2020.10 | 62.3 | 44.33 | — | 73 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CONTROL PREFIXES (A1, A2)φ%=1.4, Training Data=Augmented with DART, Control Type=Category + DART sub-dataset source2021.10 | 62.27 | — | — | — | 67.15 | 56.41 | — | — | — | — | — | — | — | — | — | — | — | |
| CONTROL PREFIXES (A1)φ%=1.4, Control Type=WebNLG category attribute2021.10 | 61.94 | — | — | — | 67.32 | 55.38 | — | — | — | — | — | — | — | — | — | — | — | |
| CONTROL PREFIXES (A2)φ%=1.0, Training Data=Augmented with DART, Control Type=DART sub-dataset source attribute2021.10 | 61.83 | — | — | — | 66.99 | 55.56 | — | — | — | — | — | — | — | — | — | — | — | |
| KGPT-SeqEncoder architecture=Seq, Pre-training=false2020.10 | 61.79 | 44.39 | — | 72.97 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Prefix-tuningφ%=1.0, Training Data=Augmented with DART2021.10 | 61.78 | — | — | — | 67.05 | 55.37 | — | — | — | — | — | — | — | — | — | — | — | |
| Prefix-tuningφ%=1.02021.10 | 61.73 | — | — | — | 66.95 | 55.39 | — | — | — | — | — | — | — | — | — | — | — | |
| SOTAφ%=1002021.10 | 61.44 | — | — | — | 65.82 | 56.01 | — | — | — | — | — | — | — | — | — | — | — | |
| Seq2Seq+CopyBackbone=sequence-to-sequence attention model, Mechanism=copy mechanism, Pre-training=false2020.10 | 61 | 42 | — | 71 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GCNEncoder architecture=Graph Convolutional Neural encoder, Pre-training=false2020.10 | 60.8 | 42.76 | — | 71.13 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ADAPTEncoder=ADAPT2018.10 | 60.6 | 44 | 37 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| T5-large fine-tunedφ%=1002021.10 | 59.95 | — | — | — | 64.89 | 54.01 | — | — | — | — | — | — | — | — | — | — | — | |
| T5Size=Large2020.05 | 57.1 | 44 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Seq2Seq+DelexBackbone=sequence-to-sequence attention model, Mechanism=delexicalization, Pre-training=false2020.10 | 56 | 39 | — | 67 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GCN_ECEncoder=GCN_EC2018.10 | 55.9 | 39 | 41 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| T5Size=Base2020.05 | 55.2 | 43 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LoRA-nBARBackbone=GPT2 (345M), # param=0.35M2024.10 | 55.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LoRA-oBARBackbone=GPT2 (345M), # param=0.35M2024.10 | 55.15 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PrefixBackbone=GPT2 (345M), # param=0.35M2024.10 | 55.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LoRABackbone=GPT2 (345M), # param=0.35M2024.10 | 54.99 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MELBOURNEEncoder=MELBOURNE2018.10 | 54.5 | 41 | 40 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| T5Size=3B2020.05 | 54 | 43 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Seq2SeqBackbone=sequence-to-sequence attention model, Pre-training=false2020.10 | 54 | 37 | — | 64 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GCNEncoder=GCN2018.10 | 53.5 | 39 | 44 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LSTMEncoder=LSTM2018.10 | 52.6 | 38 | 43 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| T5Size=Small2020.05 | 52 | 41 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Pipeline-Transformer2020.05 | 51.7 | 32 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DualEnc2020.05 | 51.4 | 41 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PKUWRITEREncoder=PKUWRITER2018.10 | 51.2 | 37 | 45 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Step-by-Step2020.05 | 47.4 | 39 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FTBackbone=GPT2 (345M), # param=354M2024.10 | 46.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Melbourne2020.05 | 45.1 | 37 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GTR-LSTM2020.05 | 37.1 | 31 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| KGPT-SeqEvaluation setting=Zero-shot, Pre-trained=true2020.10 | 13.86 | 20.15 | — | 30.23 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| KGPT-GraphEvaluation setting=Zero-shot, Pre-trained=true2020.10 | 13.66 | 19.17 | — | 30.22 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Template-GPT-2Evaluation setting=Zero-shot2020.10 | 0.3 | 0.5 | — | 3.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| All BaselinesEvaluation setting=Zero-shot2020.10 | 0 | 0 | — | 1.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| AUGUSTPLM encoder=BART-large, Graph encoder=2-layer R-GCN2023.06 | — | — | — | — | — | — | — | 35.01 | 19.82 | 48.02 | 1.65 | 43.65 | 0.9 | 0.92 | 0.87 | 9.16 | — | |
| BEST-ON-VALEnsemble Size=3B2024.12 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 30.3 | |
| BEST-ON-VALEnsemble Size=7B2024.12 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 48.3 | |
| RANDOMEnsemble Size=3B2024.12 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 23.4 | |
| RANDOMEnsemble Size=7B2024.12 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 44.1 | |
| SMOOTHIE-GLOBALEnsemble Size=3B2024.12 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 30.7 | |
| SMOOTHIE-GLOBALEnsemble Size=7B2024.12 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 45.9 |