Chinese Spelling Check on SIGHAN15 (test)
83.1F1 ScoreSpellGCN
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| SpellGCNscenario=FixLen2021.06 | 83.1 | — | — | — | — | — | — | — | — | — | 85.9 | 80.6 | |
| TtTscenario=FixLen, Training Set=HybirdSet2021.06 | 81.6 | — | — | — | — | — | — | — | — | 82.7 | 85.4 | 78.1 | |
| SpellGCNscenario=FixLen2021.06 | 81.3 | — | — | — | — | — | — | — | — | — | 85.4 | 77.6 | |
| DORM2023.05 | 81 | — | — | — | — | — | — | — | — | — | 77.9 | 84.3 | |
| LEAD2023.05 | 80.9 | — | — | — | — | — | — | — | — | — | 79.2 | 82.8 | |
| MLM-phonetics2023.05 | 80.2 | — | — | — | — | — | — | — | — | — | 77.5 | 83.1 | |
| TtTscenario=FixLen, Training Set=HybirdSet2021.06 | 80 | — | — | — | — | — | — | — | — | 81.5 | 85 | 75.6 | |
| DORM2023.05 | 79.6 | — | — | — | — | — | — | — | — | — | 76.6 | 82.8 | |
| PLOME2023.05 | 79.4 | — | — | — | — | — | — | — | — | — | 77.4 | 81.5 | |
| REALISE2023.05 | 79.3 | — | — | — | — | — | — | — | — | — | 77.3 | 81.3 | |
| LEAD2023.05 | 79.3 | — | — | — | — | — | — | — | — | — | 77.6 | 81.2 | |
| DCN2023.05 | 79 | — | — | — | — | — | — | — | — | — | 77.1 | 80.9 | |
| Soft-Masked BERT w/. Masked-FTFine-tuning=true, Masked-FT=true2023.05 | 79 | — | — | — | — | — | — | — | — | — | 76.3 | 81.8 | |
| MDCSpellFine-tuning=true2023.05 | 78.3 | — | — | — | — | — | — | — | — | — | 78.4 | 78.2 | |
| PHMOSpellFine-tuning=true, Additional pre-training=true2023.05 | 78.1 | — | — | — | — | — | — | — | — | — | 89.6 | 69.2 | |
| BERT w/. Masked-FTFine-tuning=true, Masked-FT=true2023.05 | 77.9 | — | — | — | — | — | — | — | — | — | 76.7 | 79.1 | |
| REALISE2023.05 | 77.8 | — | — | — | — | — | — | — | — | — | 75.9 | 79.9 | |
| REALISEFine-tuning=true, Additional pre-training=true2023.05 | 77.8 | — | — | — | — | — | — | — | — | — | 75.9 | 79.9 | |
| SpellGCN2023.05 | 77.7 | — | — | — | — | — | — | — | — | — | 74.8 | 80.7 | |
| MLM-phonetics2023.05 | 77.5 | — | — | — | — | — | — | — | — | — | 74.9 | 80.2 | |
| PLOME2023.05 | 77.2 | — | — | — | — | — | — | — | — | — | 75.3 | 79.3 | |
| PLOMEFine-tuning=true, Additional pre-training=true2023.05 | 77.2 | — | — | — | — | — | — | — | — | — | 75.3 | 79.3 | |
| DCN2023.05 | 76.3 | — | — | — | — | — | — | — | — | — | 74.5 | 78.2 | |
| DCNFine-tuning=true, Additional pre-training=true2023.05 | 76.3 | — | — | — | — | — | — | — | — | — | 74.5 | 78.2 | |
| BERTFine-tuned=true2023.05 | 76.1 | — | — | — | — | — | — | — | — | — | 74.2 | 78 | |
| SpellGCN2023.05 | 75.9 | — | — | — | — | — | — | — | — | — | 72.1 | 77.7 | |
| SpellGCNFine-tuning=true2023.05 | 75.9 | — | — | — | — | — | — | — | — | — | 72.1 | 77.7 | |
| ConfusBERTFine-tuning=true, Additional pre-training=true2023.05 | 74.4 | — | — | — | — | — | — | — | — | — | 72.7 | 76.1 | |
| SoftMask-BERTscenario=FixLen, Pre-training Dataset=500M-size paired dataset2021.06 | 73.5 | — | — | — | — | — | — | — | — | 80.9 | 73.7 | 73.2 | |
| BERTFine-tuned=true2023.05 | 73.4 | — | — | — | — | — | — | — | — | — | 71.6 | 75.3 | |
| Chunkscenario=FixLen2021.06 | 72.8 | — | — | — | — | — | — | — | — | 76.8 | 88.1 | 62 | |
| BERTFine-tuning=true2023.05 | 72.8 | — | — | — | — | — | — | — | — | — | 73 | 72.6 | |
| BERT-finetunescenario=FixLen, Training Set=HybirdSet2021.06 | 71.1 | — | — | — | — | — | — | — | — | 75.4 | 84.1 | 61.5 | |
| Soft-Masked BERTFine-tuning=true2023.05 | 70.1 | — | — | — | — | — | — | — | — | — | 67.6 | 72.8 | |
| Confusionsetscenario=FixLen2021.06 | 69.8 | — | — | — | — | — | — | — | — | — | 66.8 | 73.1 | |
| Chunkscenario=FixLen2021.06 | 69.4 | — | — | — | — | — | — | — | — | 74.6 | 87.3 | 57.6 | |
| SoftMask-BERTscenario=FixLen, Pre-training Dataset=500M-size paired dataset2021.06 | 66.4 | — | — | — | — | — | — | — | — | 77.4 | 66.7 | 66.2 | |
| BERT-finetunescenario=FixLen, Training Set=HybirdSet2021.06 | 65.1 | — | — | — | — | — | — | — | — | 71.6 | 82.2 | 53.9 | |
| Confusionsetscenario=FixLen2021.06 | 64.9 | — | — | — | — | — | — | — | — | — | 71.5 | 59.5 | |
| HanSpeller++scenario=FixLen2021.06 | 64 | — | — | — | — | — | — | — | — | 70.1 | 80.3 | 53.3 | |
| FASPellscenario=FixLen2021.06 | 63.5 | — | — | — | — | — | — | — | — | 74.2 | 67.6 | 60 | |
| FASPellscenario=FixLen2021.06 | 62.6 | — | — | — | — | — | — | — | — | 73.7 | 66.6 | 59.1 | |
| HanSpeller++scenario=FixLen2021.06 | 62.5 | — | — | — | — | — | — | — | — | 69.2 | 79.7 | 51.5 | |
| Hybirdscenario=FixLen, Training Set=HybirdSet2021.06 | 62.3 | — | — | — | — | — | — | — | — | — | 56.6 | 69.4 | |
| Transformer-s2sscenario=FixLen, Training Set=HybirdSet, beam-size=52021.06 | 60.9 | — | — | — | — | — | — | — | — | 67 | 73.1 | 52.2 | |
| Transformer-s2sscenario=FixLen, Training Set=HybirdSet, beam-size=52021.06 | 59.6 | — | — | — | — | — | — | — | — | 66.2 | 72.5 | 50.6 | |
| GPT2-finetunescenario=FixLen, Training Set=HybirdSet, beam-size=52021.06 | 59.4 | — | — | — | — | — | — | — | — | 65.1 | 70 | 51.9 | |
| GPT2-finetunescenario=FixLen, Training Set=HybirdSet, beam-size=52021.06 | 58.5 | — | — | — | — | — | — | — | — | 64.6 | 69.1 | 50.7 | |
| Hybirdscenario=FixLen, Training Set=HybirdSet2021.06 | 57.1 | — | — | — | — | — | — | — | — | — | — | — | |
| NCTU-NTUTscenario=FixLen2021.06 | 45.7 | — | — | — | — | — | — | — | — | 60.1 | 71.7 | 33.6 | |
| NTOUscenario=FixLen2021.06 | 42 | — | — | — | — | — | — | — | — | 42.2 | 42.2 | 41.8 | |
| NCTU-NTUTscenario=FixLen2021.06 | 37.5 | — | — | — | — | — | — | — | — | 56.4 | 66.3 | 26.1 | |
| NTOUscenario=FixLen2021.06 | 36.6 | — | — | — | — | — | — | — | — | 39 | 38.1 | 35.2 | |
| BERT2021.05 | — | 82.4 | 74.2 | 78 | 76.1 | 81 | 71.6 | 75.3 | 73.4 | — | — | — | |
| BERTStrategy=Baseline2021.05 | — | — | — | — | 76.3 | — | — | — | 74.4 | — | — | — | |
| BERTStrategy=Pre-trained for CSC2021.05 | — | — | — | — | 79.8 | — | — | — | 78 | — | — | — | |
| BERTStrategy=Adversarial training2021.05 | — | — | — | — | 80 | — | — | — | 78.2 | — | — | — | |
| FASpell2021.05 | — | 74.2 | 67.6 | 60 | 63.5 | 73.7 | 66.6 | 59.1 | 62.6 | — | — | — | |
| HanSpeller++2021.05 | — | 70.1 | 80.3 | 53.3 | 64 | 69.2 | 79.7 | 51.5 | 62.5 | — | — | — | |
| KUAS2021.05 | — | 53.2 | 57.5 | 24.6 | 34.4 | 51.5 | 53.7 | 21.1 | 30.3 | — | — | — | |
| LMC2021.05 | — | 54.6 | 63.8 | 21.5 | 32.1 | 52.3 | 57.9 | 16.7 | 26 | — | — | — | |
| NCTU-NTUT2021.05 | — | 60.1 | 71.7 | 33.6 | 45.7 | 56.4 | 66.3 | 26.1 | 37.5 | — | — | — | |
| NTOU2021.05 | — | 42.2 | 42.2 | 41.8 | 42 | 39 | 38.1 | 35.2 | 36.6 | — | — | — | |
| REALISE2021.05 | — | 84.7 | 77.3 | 81.3 | 79.3 | 84 | 75.9 | 79.9 | 77.8 | — | — | — | |
| Sequence Labeling2021.05 | — | — | 56.6 | 69.4 | 62.3 | — | — | — | 57.1 | — | — | — | |
| Soft-masked BERTStrategy=Baseline2021.05 | — | — | — | — | 72.4 | — | — | — | 69.6 | — | — | — | |
| Soft-masked BERTStrategy=Pre-trained for CSC2021.05 | — | — | — | — | 77.2 | — | — | — | 74.5 | — | — | — | |
| Soft-masked BERTStrategy=Adversarial training2021.05 | — | — | — | — | 76.2 | — | — | — | 73.8 | — | — | — | |
| Soft-Masked BERT2021.05 | — | 80.9 | 73.7 | 73.2 | 73.5 | 77.4 | 66.7 | 66.2 | 66.4 | — | — | — | |
| SpellGCN2021.05 | — | — | 74.8 | 80.7 | 77.7 | — | 72.1 | 77.7 | 75.9 | — | — | — | |
| SpellGCNStrategy=Baseline2021.05 | — | — | — | — | 73.6 | — | — | — | 71.5 | — | — | — | |
| SpellGCNStrategy=Pre-trained for CSC2021.05 | — | — | — | — | 79.6 | — | — | — | 77.7 | — | — | — | |
| SpellGCNStrategy=Adversarial training2021.05 | — | — | — | — | 79.6 | — | — | — | 77.8 | — | — | — |