Table-to-text generation on DART
0.3928METEORTeacher
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| TeacherRole=Teacher2023.07 | 0.3928 | — | 0.4545 | 0.6817 | 0.8304 | 0.4056 | 0.4856 | |
| FINE-TUNEBackbone=GPT-2MEDIUM2021.01 | 0.39 | 0.462 | 0.46 | 0.5 | 0.94 | 0.39 | — | |
| FINE-TUNEBackbone=GPT-2LARGE2021.01 | 0.39 | 0.47 | 0.46 | 0.51 | 0.94 | 0.4 | — | |
| PrefixBackbone=GPT-2LARGE2021.01 | 0.39 | 0.467 | 0.45 | 0.51 | 0.94 | 0.4 | — | |
| Fine-tuneBackbone=GPT-2, # Trainable Parameters=354.92M, # Total Parameters=100%2021.10 | 0.39 | 0.46 | 0.46 | — | — | — | — | |
| LoRABackbone=GPT-2, # Trainable Parameters=0.39M, # Total Parameters=100%2021.10 | 0.39 | 0.475 | 0.45 | — | — | — | — | |
| DSEEBackbone=GPT-2, # Trainable Parameters=0.39M, # Total Parameters=80%2021.10 | 0.39 | 0.475 | 0.46 | — | — | — | — | |
| GPT-2MEDIUMEvaluation Protocol=Fine-tuning2021.07 | 0.39 | 0.462 | 0.46 | 0.5 | 0.94 | 0.39 | — | |
| GPT-2LARGEEvaluation Protocol=Fine-tuning2021.07 | 0.39 | 0.47 | 0.46 | 0.51 | 0.94 | 0.4 | — | |
| HTLMEvaluation Protocol=Fine-tuning2021.07 | 0.39 | 0.472 | 0.44 | 0.51 | 0.94 | 0.4 | — | |
| GPT-2LARGEEvaluation Protocol=Prefix (0.1%)2021.07 | 0.39 | 0.467 | 0.45 | 0.51 | 0.94 | 0.4 | — | |
| HTLMEvaluation Protocol=Prefix (0.1%)2021.07 | 0.39 | 0.471 | 0.45 | 0.5 | 0.94 | 0.39 | — | |
| ADAPTER (3%)Backbone=GPT-2MEDIUM2021.01 | 0.38 | 0.452 | 0.46 | 0.5 | 0.94 | 0.39 | — | |
| PREFIX (0.1%)Backbone=GPT-2MEDIUM2021.01 | 0.38 | 0.464 | 0.46 | 0.5 | 0.94 | 0.39 | — | |
| AdaptersBackbone=GPT-2, # Trainable Parameters=11.48M, # Total Parameters=100%2021.10 | 0.38 | 0.454 | 0.46 | — | — | — | — | |
| PrefixBackbone=GPT-2, # Trainable Parameters=0.35M, # Total Parameters=100%2021.10 | 0.38 | 0.457 | 0.46 | — | — | — | — | |
| GPT-2MEDIUMEvaluation Protocol=Prefix (0.1%)2021.07 | 0.38 | 0.464 | 0.46 | 0.5 | 0.94 | 0.39 | — | |
| TVDRole=Student, Divergence=Total Variation2023.07 | 0.3788 | — | 0.4635 | 0.6736 | 0.8208 | 0.3717 | 0.4695 | |
| JSRole=Student, Divergence=Jensen-Shannon2023.07 | 0.3775 | — | 0.465 | 0.673 | 0.8193 | 0.3681 | 0.4685 | |
| KLRole=Student, Divergence=Kullback-Leibler2023.07 | 0.3745 | — | 0.4689 | 0.6707 | 0.816 | 0.3531 | 0.4624 | |
| RKLRole=Student, Divergence=Reverse Kullback-Leibler2023.07 | 0.3735 | — | 0.4791 | 0.6702 | 0.8141 | 0.3508 | 0.4563 | |
| SeqKDRole=Student2023.07 | 0.3717 | — | 0.4749 | 0.6665 | 0.8115 | 0.3288 | 0.4554 | |
| DSEEBackbone=GPT-2, # Trainable Parameters=0.39M, # Total Parameters=50%2021.10 | 0.37 | 0.434 | 0.51 | — | — | — | — | |
| Pre-distillRole=Student2023.07 | 0.3699 | — | 0.471 | 0.6675 | 0.8139 | 0.3408 | 0.456 | |
| ENGINERole=Student2023.07 | 0.3651 | — | 0.5063 | 0.662 | 0.8018 | 0.3094 | 0.444 | |
| ADAPTER (0.1%)Backbone=GPT-2MEDIUM2021.01 | 0.36 | 0.424 | 0.48 | 0.47 | 0.94 | 0.33 | — | |
| Non-distill (MLE)Role=Student2023.07 | 0.3571 | — | 0.4997 | 0.6565 | 0.7976 | 0.291 | 0.4312 | |
| FT-TOP2Backbone=GPT-2MEDIUM2021.01 | 0.34 | 0.41 | 0.56 | 0.43 | 0.93 | 0.21 | — | |
| FT-Top2Backbone=GPT-2, # Trainable Parameters=25.19M, # Total Parameters=100%2021.10 | 0.34 | 0.381 | 0.56 | — | — | — | — | |
| HTLMEvaluation Protocol=One-Shot2021.07 | 0.12 | 0.221 | 0.91 | 0.25 | 0.78 | 0.22 | — |