Tokenization on TWEEBANK V2 (test)
98.64F1 ScoreStanza
Evaluation results
| Method | Links | |
|---|---|---|
| StanzaTraining Data=TB22022.01 | 98.64 | |
| StanzaTraining Data=TB2+EWT2022.01 | 98.59 | |
| spaCyTraining Data=TB22022.01 | 98.57 | |
| bi-LSTM tokenizerHidden units=64, RNN Type=LSTM2018.04 | 98.3 | |
| twpipePipeline stage=Tokenization2018.04 | 98.3 | |
| Twpipe2022.01 | 98.3 | |
| UDPipe v1.2Training=re-trained on TWEEBANK V2 and UD_English-EWT2018.04 | 97.4 | |
| UDPipe v1.22022.01 | 97.4 | |
| Stanford CoreNLPType=rule-based2018.04 | 97.3 | |
| Stanford CoreNLP tokenizerPipeline stage=Tokenization2018.04 | 97.3 | |
| Stanford CoreNLP2022.01 | 97.3 | |
| spaCyTraining Data=TB2+EWT2022.01 | 95.57 | |
| TwokenizerType=rule-based, Version=updated version from Owoputi et al. (2013)2018.04 | 94.6 | |
| Twokenizer2022.01 | 94.6 |