Natural Language Generation on E2E (test)
89.94ROUGE-LFAA
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| FAAModel=DeepSeek R1-1.5B, # Trainable Parameters=1.64M2025.12 | 89.94 | 0.7683 | 9.69 | 0.6832 | — | — | — | 3.22 | — | — | |
| AdapterLModel=DeepSeek R1-1.5B, # Trainable Parameters=1.63M2025.12 | 89.48 | 0.8675 | 9.67 | 0.6776 | — | — | — | 3.13 | — | — | |
| FourierFTModel=DeepSeek R1-1.5B, # Trainable Parameters=0.15M2025.12 | 89.45 | 0.8642 | 9.62 | 0.6797 | — | — | — | 2.92 | — | — | |
| LoRAModel=DeepSeek R1-1.5B, # Trainable Parameters=1.21M2025.12 | 88.93 | 0.8703 | 9.66 | 0.6826 | — | — | — | 3.15 | — | — | |
| AdapterHModel=DeepSeek R1-1.5B, # Trainable Parameters=1.63M2025.12 | 88.23 | 0.8634 | 9.66 | 0.6815 | — | — | — | 2.98 | — | — | |
| FFModel=DeepSeek R1-1.5B, # Trainable Parameters=1.5B2025.12 | 88.22 | 0.8623 | 9.59 | 0.6794 | — | — | — | 3.21 | — | — | |
| FAAModel=LLaMA3-8B, # Trainable Parameters=8.73M2025.12 | 83.88 | 0.8216 | 9.72 | 0.6116 | — | — | — | 3.97 | — | — | |
| AdapterLModel=LLaMA3-8B, # Trainable Parameters=8.73M2025.12 | 83.79 | 0.8218 | 9.38 | 0.6116 | — | — | — | 3.9 | — | — | |
| AdapterHModel=LLaMA3-8B, # Trainable Parameters=8.73M2025.12 | 83.72 | 0.8179 | 9.47 | 0.6132 | — | — | — | 3.92 | — | — | |
| LoRAModel=LLaMA3-8B, # Trainable Parameters=6.47M2025.12 | 83.72 | 0.8222 | 9.67 | 0.6119 | — | — | — | 4.05 | — | — | |
| FourierFTModel=LLaMA3-8B, # Trainable Parameters=0.91M2025.12 | 83.65 | 0.8198 | 9.57 | 0.6127 | — | — | — | 3.99 | — | — | |
| FFModel=LLaMA3-8B, # Trainable Parameters=8.03B2025.12 | 83.61 | 0.8217 | 9.63 | 0.6127 | — | — | — | 3.97 | — | — | |
| FAAModel=LLaMA2-7B, # Trainable Parameters=7.27M2025.12 | 74.67 | 0.7318 | 9.33 | 0.5023 | — | — | — | 2.63 | — | — | |
| LoRAModel=LLaMA2-7B, # Trainable Parameters=5.37M2025.12 | 74.38 | 0.7241 | 9.32 | 0.5027 | — | — | — | 2.67 | — | — | |
| FFModel=LLaMA2-7B, # Trainable Parameters=6.74B2025.12 | 74.28 | 0.7244 | 9.15 | 0.5092 | — | — | — | 2.64 | — | — | |
| AdapterHModel=LLaMA2-7B, # Trainable Parameters=7.27M2025.12 | 73.94 | 0.7272 | 9.26 | 0.5033 | — | — | — | 2.62 | — | — | |
| AdapterLModel=LLaMA2-7B, # Trainable Parameters=7.27M2025.12 | 73.88 | 0.7236 | 9.15 | 0.5017 | — | — | — | 2.52 | — | — | |
| FourierFTModel=LLaMA2-7B, # Trainable Parameters=0.82M2025.12 | 73.78 | 0.7252 | 9.27 | 0.4973 | — | — | — | 2.74 | — | — | |
| VB-LoRAModel Scale=GPT-2 Large, #Params=0.13M2024.05 | 72.2 | 0.703 | 8.86 | 0.467 | — | — | — | 2.54 | — | — | |
| iMuonBackbone=GPT-2 Medium, LoRA rank=4, Optimizer momentum=None2026.05 | 72.14 | 0.7074 | 8.88 | 0.4679 | — | — | — | 2.54 | — | — | |
| RiemannionBackbone=GPT-2 Medium, LoRA rank=4, Optimizer momentum=None2026.05 | 71.99 | 0.7002 | 8.78 | 0.4679 | — | — | — | 2.52 | — | — | |
| LoRAModel Scale=GPT-2 Large, #Params=0.77M2024.05 | 71.9 | 0.701 | 8.8 | 0.467 | — | — | — | 2.52 | — | — | |
| Scaled AdamWModel=GPT-2 medium, r=42026.05 | 71.8 | 0.696 | 8.77 | 0.466 | — | — | — | 2.52 | — | — | |
| AdaPreLoRA AdamWModel=GPT-2 medium, r=42026.05 | 71.8 | 0.703 | 8.84 | 0.467 | — | — | — | 2.53 | — | — | |
| AdaPreLoRA SGDModel=GPT-2 medium, r=42026.05 | 71.7 | 0.703 | 8.84 | 0.469 | — | — | — | 2.54 | — | — | |
| LoRA-Pro AdamWModel=GPT-2 medium, r=42026.05 | 71.7 | 0.698 | 8.78 | 0.465 | — | — | — | 2.52 | — | — | |
| Factor-wise MuonBackbone=GPT-2 Medium, LoRA rank=4, Optimizer momentum=None2026.05 | 71.68 | 0.7002 | 8.81 | 0.4677 | — | — | — | 2.53 | — | — | |
| VeRAModel Scale=GPT-2 Large, #Params=0.17M2024.05 | 71.6 | 0.703 | 8.85 | 0.469 | — | — | — | 2.54 | — | — | |
| FAAModel=GPT-2 Small, # Trainable Parameters=0.12M2025.12 | 71.51 | 0.6656 | 8.51 | 0.4653 | — | — | — | 2.42 | — | — | |
| VeRAModel Scale=GPT-2 Medium, #Params=0.098M2024.05 | 71.5 | 0.701 | 8.81 | 0.466 | — | — | — | 2.5 | — | — | |
| VB-LoRAModel Scale=GPT-2 Medium, #Params=0.076M2024.05 | 71.5 | 0.7 | 8.81 | 0.466 | — | — | — | 2.52 | — | — | |
| AdaPreLoRA SGDModel=GPT-2 small, r=42026.05 | 71.5 | 0.695 | 8.77 | 0.465 | — | — | — | 2.5 | — | — | |
| LoRABackbone=GPT-2_medium, Number of Parameters=0.3M, Beam Size=102025.06 | 71.4 | 0.691 | 8.73 | 0.465 | — | — | — | 2.51 | — | — | |
| LoRAModel Scale=GPT-2 Medium, #Params=0.35M2024.05 | 71.3 | 0.689 | 8.69 | 0.464 | — | — | — | 2.51 | — | — | |
| AdaPreLoRA AdamWModel=GPT-2 small, r=42026.05 | 71.3 | 0.7 | 8.84 | 0.463 | — | — | — | 2.5 | — | — | |
| AdamWModel=GPT-2 medium, r=42026.05 | 71.3 | 0.689 | 8.69 | 0.465 | — | — | — | 2.51 | — | — | |
| FFModel=GPT-2 Small, # Trainable Parameters=123.65M2025.12 | 71.15 | 0.6581 | 8.22 | 0.4526 | — | — | — | 2.32 | — | — | |
| FTModel Scale=GPT-2 Medium, #Params=354.92M2024.05 | 71 | 0.682 | 8.62 | 0.462 | — | — | — | 2.47 | — | — | |
| GPT-2_medium (FT)Backbone=GPT-2_medium, Training Mode=Full tuning (FT), Number of Parameters=354.92M, Beam Size=102025.06 | 71 | 0.682 | 8.62 | 0.462 | — | — | — | 2.47 | — | — | |
| LoRAModel=GPT-2 Small, # Trainable Parameters=0.13M2025.12 | 70.97 | 0.6694 | 8.32 | 0.4626 | — | — | — | 2.33 | — | — | |
| Scaled AdamWModel=GPT-2 small, r=42026.05 | 70.9 | 0.695 | 8.8 | 0.462 | — | — | — | 2.48 | — | — | |
| Scaled GDModel=GPT-2 medium, r=42026.05 | 70.9 | 0.692 | 8.71 | 0.463 | — | — | — | 2.48 | — | — | |
| LoRA-Pro SGDModel=GPT-2 medium, r=42026.05 | 70.9 | 0.697 | 8.77 | 0.465 | — | — | — | 2.5 | — | — | |
| Scaled GDBackbone=GPT-2 Medium, LoRA rank=4, Optimizer momentum=None2026.05 | 70.9 | 0.692 | 8.71 | 0.463 | — | — | — | 2.48 | — | — | |
| Pragmatics2024.01 | 70.82 | 0.686 | 8.73 | 0.4525 | — | — | — | 2.37 | — | — | |
| LoRABackbone=GPT-2 medium, # Params (M)=0.3M, Beam size=152025.06 | 70.8 | 0.675 | 8.53 | 0.462 | — | — | — | 2.49 | — | — | |
| LoRMABackbone=GPT-2_medium, Number of Parameters=0.3M, Beam Size=102025.06 | 70.8 | 0.69 | 8.72 | 0.464 | — | — | — | 2.42 | — | — | |
| LoRMA+Backbone=GPT-2_medium, Number of Parameters=0.3M, Beam Size=102025.06 | 70.8 | 0.693 | 8.75 | 0.463 | — | — | — | 2.51 | — | — | |
| LoRA-Pro AdamWModel=GPT-2 small, r=42026.05 | 70.8 | 0.692 | 8.73 | 0.459 | — | — | — | 2.47 | — | — | |
| AdapterBackbone=GPT-2_medium, Number of Parameters=11.09M, Beam Size=102025.06 | 70.7 | 0.673 | 8.5 | 0.46 | — | — | — | 2.44 | — | — | |
| LoRMA+Backbone=GPT-2 medium, # Params (M)=0.3M, Beam size=152025.06 | 70.6 | 0.684 | 8.63 | 0.461 | — | — | — | 2.5 | — | — | |
| AdamWModel=GPT-2 small, r=42026.05 | 70.5 | 0.691 | 8.75 | 0.46 | — | — | — | 2.47 | — | — | |
| FourierFTModel=GPT-2 Small, # Trainable Parameters=0.017M2025.12 | 70.44 | 0.6636 | 8.37 | 0.4585 | — | — | — | 2.34 | — | — | |
| AdapterLModel=GPT-2 Small, # Trainable Parameters=0.12M2025.12 | 70.13 | 0.6677 | 8.21 | 0.4416 | — | — | — | 2.28 | — | — | |
| FTModel Scale=GPT-2 Large, #Params=774.03M2024.05 | 69.9 | 0.685 | 8.78 | 0.46 | — | — | — | 2.45 | — | — | |
| LoRA-Pro SGDModel=GPT-2 small, r=42026.05 | 69.6 | 0.684 | 8.72 | 0.455 | — | — | — | 2.43 | — | — | |
| Scaled GDModel=GPT-2 small, r=42026.05 | 69.4 | 0.685 | 8.72 | 0.455 | — | — | — | 2.4 | — | — | |
| T5-baserun_type=Our run2024.01 | 68.97 | 0.6695 | 8.59 | 0.457 | — | — | — | 2.27 | — | — | |
| EDA_CS2024.01 | 68.94 | 0.6705 | 8.51 | 0.4449 | — | — | — | 2.23 | — | — | |
| AdamWPrecision=4-bit, Backbone=GPT-2 Medium2023.09 | 68.9 | 0.678 | 8.61 | 0.458 | — | — | — | 2.35 | — | — | |
| BART-baserun_type=Our run2024.01 | 68.76 | 0.6574 | 8.46 | 0.456 | — | — | — | 2.2 | — | — | |
| AdapterHModel=GPT-2 Small, # Trainable Parameters=0.12M2025.12 | 68.75 | 0.6611 | 8.35 | 0.4439 | — | — | — | 2.39 | — | — | |
| AdamWPrecision=32-bit, Backbone=GPT-2 Medium2023.09 | 68.7 | 0.677 | 8.6 | 0.457 | — | — | — | 2.35 | — | — | |
| AdamW†Precision=8-bit, Backbone=GPT-2 Medium, Note=do not quantize optimizer states for embedding layers2023.09 | 68.7 | 0.675 | 8.59 | 0.457 | — | — | — | 2.34 | — | — | |
| FactorPrecision=4-bit, Backbone=GPT-2 Medium2023.09 | 68.6 | 0.676 | 8.59 | 0.456 | — | — | — | 2.34 | — | — | |
| Adafactor†Precision=32-bit, Backbone=GPT-2 Medium, Note=do not quantize optimizer states for embedding layers2023.09 | 68.5 | 0.672 | 8.54 | 0.456 | — | — | — | 2.32 | — | — | |
| TGEN2024.01 | 68.5 | 0.6593 | 8.6 | 0.4483 | — | — | — | 2.23 | — | — | |
| self-mem + new data + self T2Ddata_regime=30% random data per epoch2024.01 | 68.45 | 0.6555 | 8.41 | 0.4569 | — | — | — | 2.16 | — | — | |
| self-mem + new datadata_regime=30% random data per epoch2024.01 | 68.41 | 0.6511 | 8.35 | 0.4611 | — | — | — | 2.08 | — | — | |
| no self-mem 1data_regime=30% fixed data per epoch2024.01 | 68.33 | 0.6547 | 8.32 | 0.4584 | — | — | — | 2.17 | — | — | |
| AdafactorPrecision=32-bit, Backbone=GPT-2 Medium2023.09 | 68.3 | 0.672 | 8.61 | 0.453 | — | — | — | 2.35 | — | — | |
| no self-mem 3data_regime=30% random data per epoch2024.01 | 68.24 | 0.655 | 8.4 | 0.453 | — | — | — | 2.2 | — | — | |
| SM3Precision=32-bit, Backbone=GPT-2 Medium2023.09 | 68.2 | 0.669 | 8.59 | 0.454 | — | — | — | 2.33 | — | — | |
| SGDModel=GPT-2 medium, r=42026.05 | 68.2 | 0.666 | 8.54 | 0.442 | — | — | — | 2.32 | — | — | |
| SGDBackbone=GPT-2 Medium, LoRA rank=4, Optimizer momentum=None2026.05 | 68.2 | 0.666 | 8.54 | 0.442 | — | — | — | 2.32 | — | — | |
| self-mem + new datadata_regime=30% fixed data per epoch2024.01 | 68.16 | 0.6555 | 8.35 | 0.4607 | — | — | — | 2.1 | — | — | |
| self-mem + new data + self T2Ddata_regime=30% fixed data per epoch2024.01 | 68.11 | 0.6547 | 8.38 | 0.461 | — | — | — | 2.07 | — | — | |
| FLAN-T5-baserun_type=Our run2024.01 | 67.85 | 0.6565 | 8.49 | 0.4554 | — | — | — | 2.12 | — | — | |
| no self-mem 2data_regime=30% fixed data per epoch2024.01 | 67.76 | 0.6494 | 8.33 | 0.4513 | — | — | — | 2.21 | — | — | |
| SLUG2024.01 | 67.72 | 0.6619 | 8.61 | 0.4454 | — | — | — | — | — | — | |
| self-memdata_regime=30% fixed data per epoch2024.01 | 66.58 | 0.6128 | 8.05 | 0.4484 | — | — | — | 2.05 | — | — | |
| self-mem + self T2Ddata_regime=30% random data per epoch2024.01 | 66.54 | 0.6174 | 8.07 | 0.4489 | — | — | — | 2.11 | — | — | |
| self-mem + self T2Ddata_regime=30% fixed data per epoch2024.01 | 66.45 | 0.6069 | 8.03 | 0.4448 | — | — | — | 2.03 | — | — | |
| self-memdata_regime=30% random data per epoch2024.01 | 66.44 | 0.6198 | 8.05 | 0.4448 | — | — | — | 2.14 | — | — | |
| SGDModel=GPT-2 small, r=42026.05 | 63.3 | 0.548 | 4.56 | 0.34 | — | — | — | 1.29 | — | — | |
| GENICLBackbone=LLaMA-3.2-3B2025.05 | 57.5 | — | — | — | — | — | — | — | — | — | |
| LLM-RBackbone=LLaMA-3.2-3B2025.05 | 56.4 | — | — | — | — | — | — | — | — | — | |
| GENICLBackbone=Vicuna-13B2025.05 | 56.3 | — | — | — | — | — | — | — | — | — | |
| LLM-RBackbone=Vicuna-13B2025.05 | 55.4 | — | — | — | — | — | — | — | — | — | |
| Z-Code++Number of Parameters=800M2022.08 | 54 | — | — | — | — | — | — | — | 74.8 | 46.9 | |
| GENICLBackbone=GPT-Neo 2.7B2025.05 | 50.1 | — | — | — | — | — | — | — | — | — | |
| T5-XLargeNumber of Parameters=3B2022.08 | 49.7 | — | — | — | — | — | — | — | 70.8 | 41.7 | |
| T5-LargeNumber of Parameters=800M2022.08 | 49.5 | — | — | — | — | — | — | — | 70.8 | 41.7 | |
| LLM-RBackbone=GPT-Neo 2.7B2025.05 | 49.2 | — | — | — | — | — | — | — | — | — | |
| GENICLBackbone=Qwen2.5-3B2025.05 | 37.9 | — | — | — | — | — | — | — | — | — | |
| LLM-RBackbone=Qwen2.5-3B2025.05 | 37 | — | — | — | — | — | — | — | — | — | |
| Zero-shotBackbone=LLaMA-3.2-3B2025.05 | 33.5 | — | — | — | — | — | — | — | — | — | |
| Zero-shotBackbone=Qwen2.5-3B2025.05 | 13.6 | — | — | — | — | — | — | — | — | — | |
| Zero-shotBackbone=GPT-Neo 2.7B2025.05 | 7.71 | — | — | — | — | — | — | — | — | — | |
| Zero-shotBackbone=Vicuna-13B2025.05 | 1.71 | — | — | — | — | — | — | — | — | — |