Legal Language Understanding on LexGLUE (test)
78.1CaseHOLD Macro F1CompassMTL
Evaluation results
| Method | Links | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| CompassMTLtailor=true2022.10 | 78.1 | — | 64.7 | — | 72.3 | — | 68.6 | — | 44.9 | — | 83.2 | — | 83.2 | — | |
| CompassMTLtailor=false2022.10 | 76.1 | — | 60.7 | — | 73.2 | — | 68.9 | — | 42.1 | — | 82.3 | — | 84.3 | — | |
| CaseLaw-BERT2022.10 | 75.4 | — | 62.9 | — | 70.3 | — | 65.9 | — | 56.6 | — | 83 | — | 82.3 | — | |
| Legal-BERT2022.10 | 75.3 | — | 64 | — | 74.7 | — | 66.5 | — | 57.4 | — | 83 | — | 83 | — | |
| ExDeBERTa2022.10 | 74.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DeBERTa2022.10 | 72.6 | — | 60.8 | — | 71 | — | 62.7 | — | 57.4 | — | 83.1 | — | 80.3 | — | |
| Longformer2022.10 | 71.9 | — | 64.7 | — | 71.7 | — | 64 | — | 57.7 | — | 83 | — | 80.9 | — | |
| RoBERTa2022.10 | 71.4 | — | 59 | — | 68.9 | — | 62 | — | 57.9 | — | 82.3 | — | 79.2 | — | |
| BERT2022.10 | 70.8 | — | 63.6 | — | 73.4 | — | 58.3 | — | 57.2 | — | 81.8 | — | 81.3 | — | |
| BigBird2022.10 | 70.8 | — | 62.9 | — | 70.9 | — | 62 | — | 56.8 | — | 82.6 | — | 81.3 | — |