Semantic Parsing on COGS (test)
99Exact Match AccuracyFull FT
Evaluation Results
| Method | Links | |
|---|---|---|
| Full FTCategory=No protection, Best Ep=6.02026.06 | 99 | |
| WProjCategory=Weight-space, Best Ep=4.42026.06 | 98.92 | |
| EWCCategory=Regularization, Best Ep=6.02026.06 | 98.84 | |
| ForaCategory=Ours, Best Ep=4.82026.06 | 98.68 | |
| FProjCategory=Ours, Best Ep=5.02026.06 | 98.64 | |
| L2-SPCategory=Regularization, Best Ep=5.02026.06 | 98.48 | |
| SpanSub+L2S2base_model=LSTM2023.06 | 92.3 | |
| LwFCategory=Regularization, Best Ep=8.22026.06 | 92.12 | |
| SpanSubbase_model=LSTM2023.06 | 91.8 | |
| Roberta+Dangle2023.06 | 87.6 | |
| T5-Base2023.06 | 85.9 | |
| Probabilistic Transformer2023.11 | 84.6 | |
| SQ-TransformerAttention Protocol=SRL2024.02 | 83.36 | |
| TransformerAttention Protocol=SRL2024.02 | 82.6 | |
| Transformer2023.11 | 82.05 | |
| Lex Learn2024.02 | 82 | |
| LexLearnbase_model=LSTM2023.06 | 82 | |
| LexSymbase_model=LSTM2023.06 | 81.4 | |
| Prim2PrimX+METbase_model=LSTM2023.06 | 81.1 | |
| IR-Transformer2023.06 | 78.4 | |
| MAML2023.06 | 64.1 | |
| LSTM2023.06 | 55.4 | |
| GECAbase_model=LSTM2023.06 | 48 |