Natural Language Understanding on GLUE (val) (CoLA and MRPC Metrics)
85.7MRPC AccuracyTyphoon
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| TyphoonBackbone=BERT-base2023.03 | 85.7 | 90.1 | 0.577 | 82.7 | |
| Random maskingBackbone=BERT-base2023.03 | 85.5 | 90 | 0.58 | 82.8 | |
| Whole-word maskingBackbone=BERT-base2023.03 | 85.3 | 89.7 | 0.581 | 82.9 | |
| Whole-word maskingBackbone=DistilBERT-base2023.03 | 85.2 | 89.9 | 0.504 | 79.8 | |
| TyphoonBackbone=DistilBERT-base2023.03 | 84.9 | 89.5 | 0.509 | 79.8 | |
| Random maskingBackbone=DistilBERT-base2023.03 | 84.5 | 89.2 | 0.512 | 79.9 | |
| Random maskingBackbone=TinyBERT2023.03 | 83.3 | 88.5 | 0.148 | 69.7 | |
| TyphoonBackbone=TinyBERT2023.03 | 83.3 | 88.5 | 0.148 | 69.5 | |
| Whole-word maskingBackbone=TinyBERT2023.03 | 83.2 | 88.5 | 0.162 | 70 |