AMR Parsing on BioAMR (test)
81.3Smatch ScoreStruct-BART + MBSE-G silver + gold
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Struct-BART + MBSE-G silver + goldTraining Data=AMR2.0 + MBSE-G silv.b + 5K gold2021.12 | 81.3 | 81 | |
| SPRING-DFS (In domain)Training Data=SPRING-DFS trained on in-domain data2021.12 | 79.9 | — | |
| Struct-BART + gold sentsTraining Data=AMR2.0 + 5K domain gold sents2021.12 | 79.8 | 80 | |
| Struct-BART + MBSE-G silver + goldTraining Data=AMR2.0 + MBSE-G silv.1 + 201 gold2021.12 | 75.8 | 70 | |
| Struct-BART + gold sentsTraining Data=AMR2.0 + 403 domain gold sents2021.12 | 74.3 | 70 | |
| Struct-BART + gold sentsTraining Data=AMR2.0 + 201 domain gold sents2021.12 | 72.3 | 68 | |
| Struct-BART + MBSE-G silverTraining Data=AMR2.0 + MBSE-G silver (Graphene-Smatch)2021.12 | 66.9 | 30 | |
| Struct-BART + MBSE-P silverTraining Data=AMR2.0 + MBSE-P silver (Graphene-Smatch)2021.12 | 66.9 | 31 | |
| LeakDistillevaluation_setting=out-of-distribution2023.06 | 64.5 | — | |
| AMRBARTevaluation_setting=out-of-distribution2023.06 | 63.4 | — | |
| Struct-BART + MBSE-G silverTraining Data=AMR2.0 + MBSE-G silver (reference 1)2021.12 | 63.2 | 31 | |
| SPRINGevaluation_setting=out-of-distribution2023.06 | 61.6 | — | |
| ATPevaluation_setting=out-of-distribution2023.06 | 61.2 | — | |
| BiBLevaluation_setting=out-of-distribution2023.06 | 61.1 | — | |
| Struct-BARTTraining Data=AMR2.02021.12 | 60.4 | 27 | |
| SPRING-DFSTraining Data=Standard SPRING-DFS2021.12 | 59.7 | — | |
| SPRING-DFS + silverTraining Data=SPRING-DFS with silver data2021.12 | 59.5 | — |