Probing on LegalLAMA 1.0 (test)
94.3ECHR Articles ScoreLexLM
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| LexLMModel Size=Large, Training Set=LEXFILES2023.05 | 94.3 | 87.3 | 86.1 | 68.1 | 67.5 | 70.4 | 86.8 | 68.8 | 77.4 | 1 | |
| LexLMModel Size=Base, Training Set=LEXFILES2023.05 | 91.4 | 88.2 | 84 | 63 | 66.4 | 63.1 | 81.3 | 50.6 | 70.8 | 2 | |
| LegalBERTTraining Data=UK, EU and USA legal data2023.05 | 91.1 | 80.2 | 82.2 | 51.5 | 60.5 | 63.2 | 77.3 | 21.9 | 54.8 | 3 | |
| ROBERTaModel Size=Large2023.05 | 41.3 | 44.5 | 47.8 | 62.4 | 54.2 | 45.3 | 53.1 | 19.7 | 41.3 | 4 | |
| ROBERTaModel Size=Base2023.05 | 39.8 | 23.6 | 43.4 | 56.3 | 47.1 | 38 | 45.4 | 15.8 | 33.1 | 7 | |
| CL-BERTFull Name=CaseLaw BERT, Domain=US legal2023.05 | 37.5 | 29.2 | 54.9 | 62.6 | 66.7 | 38.6 | 49.7 | 18.4 | 38 | 5 | |
| POL-BERTDomain=US legal2023.05 | 35.2 | 64.8 | 49.7 | 43.5 | 44.6 | 36.9 | 32.8 | 19.9 | 36.8 | 6 |