Table-to-text Generation on E2ENLG (test)
70.74BLEUiMuon
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| iMuonMomentum=false, Backbone=GPT-2 Medium, LoRA rank=42026.05 | 70.74 | — | 8.88 | 46.79 | 2.54 | — | — | 72.14 | — | |
| iMuonMomentum=true, Backbone=GPT-2 Medium, LoRA rank=42026.05 | 70.36 | — | 8.82 | 46.75 | 2.53 | — | — | 71.92 | — | |
| Factor-wise MuonMomentum=true, Backbone=GPT-2 Medium, LoRA rank=42026.05 | 70.26 | — | 8.83 | 46.76 | 2.52 | — | — | 71.75 | — | |
| Riemannion (Adam)Momentum=true, Backbone=GPT-2 Medium, LoRA rank=42026.05 | 70.21 | — | 8.83 | 46.55 | 2.53 | — | — | 71.87 | — | |
| Factor-wise MuonMomentum=false, Backbone=GPT-2 Medium, LoRA rank=42026.05 | 70.02 | — | 8.81 | 46.77 | 2.53 | — | — | 71.68 | — | |
| Riemannion (SGD)Momentum=false, Backbone=GPT-2 Medium, LoRA rank=42026.05 | 70.02 | — | 8.78 | 46.79 | 2.52 | — | — | 71.99 | — | |
| Scaled AdamWMomentum=true, Backbone=GPT-2 Medium, LoRA rank=42026.05 | 69.6 | — | 8.77 | 46.6 | 2.52 | — | — | 71.8 | — | |
| Scaled GDMomentum=false, Backbone=GPT-2 Medium, LoRA rank=42026.05 | 69.2 | — | 8.71 | 46.3 | 2.48 | — | — | 70.9 | — | |
| AdamWMomentum=true, Backbone=GPT-2 Medium, LoRA rank=42026.05 | 68.9 | — | 8.69 | 46.5 | 2.51 | — | — | 71.3 | — | |
| SGDMomentum=false, Backbone=GPT-2 Medium, LoRA rank=42026.05 | 66.6 | — | 8.54 | 44.2 | 2.32 | — | — | 68.2 | — | |
| NEUROLOGIC*Base Model=GPT-2, Fine-tuning Data=0.1%, Decoding Method=sample2021.12 | 49.3 | — | 7.11 | 40.1 | 17.5 | 60 | 100 | — | — | |
| NEUROLOGIC*Base Model=GPT-2, Fine-tuning Data=0.1%, Decoding Method=greedy2021.12 | 49.2 | — | 7.11 | 40 | 17.5 | 60 | 100 | — | — | |
| NEUROLOGIC*Base Model=GPT-2, Fine-tuning Data=0.1%, Decoding Method=beam2021.12 | 48.9 | — | 7.01 | 40 | 17.2 | 59.8 | 99.9 | — | — | |
| NEUROLOGICBase Model=GPT-2, Fine-tuning Data=0.1%, Decoding Method=NEUROLOGIC2021.12 | 47.6 | — | 6.95 | 38.9 | 16.3 | 58.7 | 97.6 | — | — | |
| Beam SearchBase Model=GPT-2, Fine-tuning Data=0.1%, Decoding Method=Beam Search2021.12 | 42.8 | — | 3.82 | 32.6 | 10.8 | 57.8 | 73.6 | — | — | |
| CBSBase Model=GPT-2, Fine-tuning Data=0.1%, Decoding Method=CBS2021.12 | 42.3 | — | 6.5 | 36.4 | 13 | 54.3 | 91.6 | — | — | |
| GBSBase Model=GPT-2, Fine-tuning Data=0.1%, Decoding Method=GBS2021.12 | 40.7 | — | 6.26 | 36.7 | 12.9 | 54.2 | 94.1 | — | — | |
| DP-MuonBCPrivacy budget (epsilon)=8, Model=GPT-2, Number of seeds=32026.05 | 34.92 | — | — | — | — | — | — | 37.47 | 0.449 | |
| DP-MuonPrivacy budget (epsilon)=8, Model=GPT-2, Number of seeds=32026.05 | 32.91 | — | — | — | — | — | — | 37.64 | 0.4553 | |
| DP-AdamPrivacy budget (epsilon)=8, Model=GPT-2, Number of seeds=32026.05 | 18.49 | — | — | — | — | — | — | 31.94 | 0.4961 | |
| DP-SGDPrivacy budget (epsilon)=8, Model=GPT-2, Number of seeds=32026.05 | 3.42 | — | — | — | — | — | — | 28.55 | 0.6237 | |
| LoRABackbone=GPT2 (124M), DP guarantee (epsilon)=non-DP2022.06 | 0.6968 | — | — | — | — | — | — | 71.709 | — | |
| AUTO-SBackbone=GPT2 (124M), DP guarantee (epsilon)=non-DP2022.06 | 0.6946 | — | — | — | — | — | — | 71.359 | — | |
| AUTO-VBackbone=GPT2 (124M), DP guarantee (epsilon)=non-DP2022.06 | 0.6946 | — | — | — | — | — | — | 71.359 | — | |
| fullBackbone=GPT2 (124M), DP guarantee (epsilon)=non-DP2022.06 | 0.6946 | — | — | — | — | — | — | 71.359 | — | |
| prefixBackbone=GPT2 (124M), DP guarantee (epsilon)=non-DP2022.06 | 0.6885 | — | — | — | — | — | — | 70.805 | — | |
| AUTO-SBackbone=GPT2 medium (355M), DP guarantee (epsilon)=non-DP2022.06 | 0.685 | — | — | — | — | — | — | 71.458 | — | |
| RGPBackbone=GPT2 (124M), DP guarantee (epsilon)=non-DP2022.06 | 0.6833 | — | — | — | — | — | — | 68.844 | — | |
| AUTO-SBackbone=GPT2 large (774M), DP guarantee (epsilon)=non-DP2022.06 | 0.6684 | — | — | — | — | — | — | 70.384 | — | |
| top2Backbone=GPT2 (124M), DP guarantee (epsilon)=non-DP2022.06 | 0.6575 | — | — | — | — | — | — | 68.704 | — | |
| retrainBackbone=GPT2 (124M), DP guarantee (epsilon)=non-DP2022.06 | 0.6573 | — | — | — | — | — | — | 68.751 | — | |
| AUTO-SBackbone=GPT2 large (774M), DP guarantee (epsilon)=82022.06 | 0.6464 | — | — | — | — | — | — | 68.968 | — | |
| AUTO-SBackbone=GPT2 medium (355M), DP guarantee (epsilon)=82022.06 | 0.6422 | — | — | — | — | — | — | 67.533 | — | |
| AUTO-SBackbone=GPT2 large (774M), DP guarantee (epsilon)=32022.06 | 0.6418 | — | — | — | — | — | — | 67.857 | — | |
| AUTO-SBackbone=GPT2 medium (355M), DP guarantee (epsilon)=32022.06 | 0.6385 | — | — | — | — | — | — | 67.071 | — | |
| AUTO-SBackbone=GPT2 (124M), DP guarantee (epsilon)=82022.06 | 0.636 | — | — | — | — | — | — | 67.073 | — | |
| LoRABackbone=GPT2 (124M), DP guarantee (epsilon)=82022.06 | 0.6339 | — | — | — | — | — | — | 67.525 | — | |
| AUTO-VBackbone=GPT2 (124M), DP guarantee (epsilon)=82022.06 | 0.6319 | — | — | — | — | — | — | 66.429 | — | |
| fullBackbone=GPT2 (124M), DP guarantee (epsilon)=82022.06 | 0.6319 | — | — | — | — | — | — | 66.429 | — | |
| AUTO-VBackbone=GPT2 (124M), DP guarantee (epsilon)=32022.06 | 0.6152 | — | — | — | — | — | — | 65.67 | — | |
| fullBackbone=GPT2 (124M), DP guarantee (epsilon)=32022.06 | 0.6152 | — | — | — | — | — | — | 65.67 | — | |
| AUTO-SBackbone=GPT2 (124M), DP guarantee (epsilon)=32022.06 | 0.6134 | — | — | — | — | — | — | 65.872 | — | |
| RGPBackbone=GPT2 (124M), DP guarantee (epsilon)=32022.06 | 0.5848 | — | — | — | — | — | — | 65.56 | — | |
| RGPBackbone=GPT2 (124M), DP guarantee (epsilon)=82022.06 | 0.5846 | — | — | — | — | — | — | 65.03 | — | |
| LoRABackbone=GPT2 (124M), DP guarantee (epsilon)=32022.06 | 0.5815 | — | — | — | — | — | — | 65.773 | — | |
| prefixBackbone=GPT2 (124M), DP guarantee (epsilon)=82022.06 | 0.4926 | — | — | — | — | — | — | 60.73 | — | |
| prefixBackbone=GPT2 (124M), DP guarantee (epsilon)=32022.06 | 0.4777 | — | — | — | — | — | — | 58.964 | — | |
| top2Backbone=GPT2 (124M), DP guarantee (epsilon)=82022.06 | 0.2689 | — | — | — | — | — | — | 46.421 | — | |
| top2Backbone=GPT2 (124M), DP guarantee (epsilon)=32022.06 | 0.2592 | — | — | — | — | — | — | 44.536 | — | |
| retrainBackbone=GPT2 (124M), DP guarantee (epsilon)=82022.06 | 0.2425 | — | — | — | — | — | — | 39.951 | — | |
| retrainBackbone=GPT2 (124M), DP guarantee (epsilon)=32022.06 | 0.1546 | — | — | — | — | — | — | 35.24 | — | |
| GPT-2training_data_percentage=0.1%2021.12 | — | 42.8 | — | — | — | — | — | — | — | |
| GPT-2training_data_percentage=0.5%2021.12 | — | 57.1 | — | — | — | — | — | — | — | |
| GPT-2training_data_percentage=1%2021.12 | — | 56.8 | — | — | — | — | — | — | — | |
| GPT-2training_data_percentage=5%2021.12 | — | 61.1 | — | — | — | — | — | — | — | |
| GPT-2 + NEUROLOGICtraining_data_percentage=0.1%2021.12 | — | 47.6 | — | — | — | — | — | — | — | |
| GPT-2 + NEUROLOGICtraining_data_percentage=0.5%2021.12 | — | 56.9 | — | — | — | — | — | — | — | |
| GPT-2 + NEUROLOGICtraining_data_percentage=1%2021.12 | — | 58 | — | — | — | — | — | — | — | |
| GPT-2 + NEUROLOGICtraining_data_percentage=5%2021.12 | — | 62.9 | — | — | — | — | — | — | — | |
| GPT-2 + NEUROLOGIC* (greedy)training_data_percentage=0.1%, Decoding strategy=greedy2021.12 | — | 49.2 | — | — | — | — | — | — | — | |
| GPT-2 + NEUROLOGIC* (greedy)training_data_percentage=0.5%, Decoding strategy=greedy2021.12 | — | 58 | — | — | — | — | — | — | — | |
| GPT-2 + NEUROLOGIC* (greedy)training_data_percentage=1%, Decoding strategy=greedy2021.12 | — | 58.4 | — | — | — | — | — | — | — | |
| GPT-2 + NEUROLOGIC* (greedy)training_data_percentage=5%, Decoding strategy=greedy2021.12 | — | 63.4 | — | — | — | — | — | — | — | |
| KGPT-Graphtraining_data_percentage=0.1%2021.12 | — | 39.8 | — | — | — | — | — | — | — | |
| KGPT-Graphtraining_data_percentage=0.5%2021.12 | — | 53.3 | — | — | — | — | — | — | — | |
| KGPT-Graphtraining_data_percentage=1%2021.12 | — | 55.1 | — | — | — | — | — | — | — | |
| KGPT-Graphtraining_data_percentage=5%2021.12 | — | 61.5 | — | — | — | — | — | — | — | |
| KGPT-Seqtraining_data_percentage=0.1%2021.12 | — | 40.2 | — | — | — | — | — | — | — | |
| KGPT-Seqtraining_data_percentage=0.5%2021.12 | — | 53 | — | — | — | — | — | — | — | |
| KGPT-Seqtraining_data_percentage=1%2021.12 | — | 54.1 | — | — | — | — | — | — | — | |
| KGPT-Seqtraining_data_percentage=5%2021.12 | — | 61.1 | — | — | — | — | — | — | — | |
| Template-GPT-2training_data_percentage=0.1%2021.12 | — | 22.5 | — | — | — | — | — | — | — | |
| Template-GPT-2training_data_percentage=0.5%2021.12 | — | 47.8 | — | — | — | — | — | — | — | |
| Template-GPT-2training_data_percentage=1%2021.12 | — | 53.3 | — | — | — | — | — | — | — | |
| Template-GPT-2training_data_percentage=5%2021.12 | — | 59.9 | — | — | — | — | — | — | — | |
| TGentraining_data_percentage=0.1%2021.12 | — | 3.6 | — | — | — | — | — | — | — | |
| TGentraining_data_percentage=0.5%2021.12 | — | 27.9 | — | — | — | — | — | — | — | |
| TGentraining_data_percentage=1%2021.12 | — | 35.2 | — | — | — | — | — | — | — | |
| TGentraining_data_percentage=5%2021.12 | — | 57.3 | — | — | — | — | — | — | — |