Machine Translation on CoGnition compositional generalization (test)
16Inst. Error RateR-Dangle_sep
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| R-Dangle_sepinterval=12022.12 | 16 | 42.1 | 63.4 | |
| SQ-TransformerVariant=SRL2024.02 | 18.14 | 48.89 | 62.78 | |
| Dangleimplementation source=our implementation2022.12 | 19.7 | 47 | 62.5 | |
| Consistency-Reg2024.02 | 20.2 | 48.3 | 61.3 | |
| Proto-Transformer2024.02 | 21.7 | 51.8 | 60.1 | |
| DANGLE-ENCDECPosition Embedding=Relative2021.10 | 22.8 | 50.6 | 60.6 | |
| Dangleimplementation source=Zheng and Lapata, 20222022.12 | 22.8 | 50.6 | 60.6 | |
| Dangle-Transformer2024.02 | 22.8 | 50.6 | 60.6 | |
| Transformerimplementation source=our implementation2022.12 | 23.4 | 53.7 | 61.9 | |
| DANGLE-ENCDECPosition Embedding=Absolute2021.10 | 24.4 | 55.5 | 59.7 | |
| TransformerSource=Previous Models2024.02 | 28.4 | 62.9 | 59.5 | |
| TRANSFORMERPosition Embedding=Absolute2021.10 | 29.4 | 63.8 | 59.4 | |
| TransformerSource=Our Models2024.02 | 29.55 | 61.62 | 60.45 | |
| TRANSFORMERPosition Embedding=Relative2021.10 | 30.5 | 63.8 | 59.4 | |
| Transformerimplementation source=Zheng and Lapata, 20222022.12 | 30.5 | 63.8 | 59.4 |