Coherence Classification on GCDC 1.0 (test)
66.9Clinton F1MTL with Transformers
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| MTL with Transformerstraining_data=single domain2021.09 | 66.9 | 55.1 | 58.9 | 56.6 | 59.4 | |
| Hierarchical Transformertraining_data=all domains2021.09 | 66.5 | 56.3 | 60.9 | 58.5 | 60.6 | |
| MTL with Transformertraining_data=all domains2021.09 | 66.5 | 56.4 | 62.2 | 58.9 | 61 | |
| Hierarchical Transformertraining_data=single domain2021.09 | 66 | 55.4 | 60.7 | 59.2 | 60.3 | |
| Vanilla Transformertraining_data=single domain2021.09 | 65.6 | 56.2 | 59.3 | 57.4 | 59.6 | |
| Fact-aware Transformertraining_data=single domain2021.09 | 64.3 | 53.8 | 58.4 | 58.1 | 58.7 | |
| Vanilla Transformertraining_data=all domains2021.09 | 63.9 | 55.3 | 58.1 | 57.6 | 58.7 | |
| Coh+GRtraining_data=single domain2021.09 | 62 | 56 | 56 | — | 58 | |
| LCtraining_data=single domain2021.09 | 61 | 54.4 | 53.5 | — | 56.3 | |
| PARSEQtraining_data=all domains2021.09 | 61 | 53.9 | 58.5 | 56.5 | 57.5 | |
| PARSEQtraining_data=single domain2021.09 | 60.2 | 53.2 | 54.9 | 54.4 | 55.7 | |
| Hierachical LSTMtraining_data=single domain2021.09 | 59 | 50.5 | 55 | — | 54.8 | |
| Coh+SOXtraining_data=single domain2021.09 | 58.5 | 51 | 50.5 | — | 53.3 | |
| Lexicalized CNN-Egrid+coreftraining_data=single domain2021.09 | 56.6 | 44.7 | 51 | 54 | 51.6 | |
| Flesch-Kincaid grade leveltraining_data=single domain2021.09 | 56 | 52.5 | 43.5 | 55 | 51.8 | |
| EGRAPH+coreftraining_data=single domain2021.09 | 55 | 44 | 42.5 | 54 | 48.9 | |
| Ensemble (ParSeq + RST-Recursive)T (RST Tree structure)=true, NS (Nuclearity)=true, R (Relations)=true, E (EDU embeddings)=false, EDU-level semantic information via ParSeq=true2020.09 | 52.42 | 44.69 | 46.88 | 43.94 | 46.98 | |
| Ensemble (ParSeq + RST-Recursive)T (RST Tree structure)=true, NS (Nuclearity)=false, R (Relations)=false, E (EDU embeddings)=false, EDU-level semantic information via ParSeq=true2020.09 | 52.35 | 44.92 | 45.48 | 43.7 | 46.61 | |
| ParSeq2020.09 | 52.12 | 44.9 | 46.22 | 43.36 | 46.65 | |
| Ensemble (ParSeq + RST-Recursive)T (RST Tree structure)=true, NS (Nuclearity)=true, R (Relations)=false, E (EDU embeddings)=false, EDU-level semantic information via ParSeq=true2020.09 | 51.9 | 44.76 | 45.48 | 43.83 | 46.49 | |
| EGRID+coreftraining_data=single domain2021.09 | 48 | 47 | 41.5 | 49 | 46.4 | |
| RST-RecursiveT (RST Tree structure)=true, NS (Nuclearity)=true, R (Relations)=true, E (EDU embeddings)=true2020.09 | 45.9 | 44.33 | 43.85 | 43.13 | 44.3 | |
| RST-RecursiveT (RST Tree structure)=true, NS (Nuclearity)=true, R (Relations)=true, E (EDU embeddings)=false2020.09 | 41.08 | 31.21 | 41.97 | 42.27 | 39.13 | |
| Majority2020.09 | 39.42 | 27.29 | 20.95 | 38.82 | 31.62 | |
| RST-RecursiveT (RST Tree structure)=true, NS (Nuclearity)=false, R (Relations)=false, E (EDU embeddings)=false2020.09 | 39.42 | 27.29 | 20.95 | 38.82 | 31.62 | |
| RST-RecursiveT (RST Tree structure)=true, NS (Nuclearity)=true, R (Relations)=false, E (EDU embeddings)=false2020.09 | 39.2 | 30.81 | 35.67 | 39.93 | 36.4 |