Commonsense Reasoning on WinoGender
0.9681AccuracyFrozen knowledge
Evaluation results
| Method | Links | |
|---|---|---|
| Frozen knowledgeLM=RoBERTa, Avg.=80.01, Speed-up=1.5x2023.05 | 0.9681 | |
| Batch PartitioningLM=RoBERTa, Avg.=85.14, Speed-up=1.4x2023.05 | 0.9625 | |
| OK-TransformerLM=RoBERTa, Avg.=84.75, Speed-up=1.0x2023.05 | 0.95 | |
| RoBERTaLM=RoBERTa, Avg.=83.952023.05 | 0.946 | |
| Batch PartitioningLM=BERT, Avg.=67.40, Speed-up=1.4x2023.05 | 0.7278 | |
| OK-TransformerLM=BERT, Avg.=66.56, Speed-up=1.0x2023.05 | 0.7153 | |
| Frozen knowledgeLM=BERT, Avg.=65.59, Speed-up=1.4x2023.05 | 0.6847 | |
| BERTLM=BERT, Avg.=65.442023.05 | 0.6819 |