Clause Classification on Illegal Clauses
83Macro F1BETO cased (Spanish)
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| BETO cased (Spanish)Casing=cased, Language=Spanish2026.05 | 83 | 95 | |
| BERT cased (multiling.)Casing=cased, Language=multilingual2026.05 | 81 | 94 | |
| BERT uncased (multiling.)Casing=uncased, Language=multilingual2026.05 | 81 | 94 | |
| BETO uncased (Spanish)Casing=uncased, Language=Spanish2026.05 | 81 | 94 | |
| RoBERTalex (Spanish, legal)Language=Spanish, Domain=legal2026.05 | 80 | 94 | |
| RoBERTa2026.05 | 80 | 94 | |
| RoBERTa (multiling.)Language=multilingual2026.05 | 80 | 94 | |
| Majority-Vote (Dense)Retrieval Mode=Dense2026.05 | 79 | 94 | |
| BERT casedCasing=cased2026.05 | 79 | 94 | |
| BERT uncasedCasing=uncased2026.05 | 79 | 94 | |
| Longformer2026.05 | 79 | 94 | |
| SVM (TF-IDF)Representation=TF-IDF2026.05 | 78 | 94 | |
| Majority-Vote (Hybrid)Retrieval Mode=Hybrid2026.05 | 78 | 94 | |
| BERT uncased (legal)Casing=uncased, Domain=legal2026.05 | 77 | 92 | |
| Qwen3 14b (Hybrid)Parameters=14b, Retrieval Mode=Hybrid2026.05 | 75 | 91 | |
| RoBERTa (multiling., large)Language=multilingual, Size=large2026.05 | 66 | 93 | |
| GPT 5 nanoRetrieval Type=Dense2026.05 | 56 | 68 | |
| GPT 5.2Retrieval Type=Hybrid2026.05 | 55 | 72 | |
| GPT 5.2Retrieval Type=Dense2026.05 | 55 | 71 | |
| Qwen3 14bRetrieval Type=Hybrid2026.05 | 55 | 71 | |
| GPT 5 miniRetrieval Type=Hybrid2026.05 | 53 | 71 | |
| GPT 5 nanoRetrieval Type=Hybrid2026.05 | 53 | 67 | |
| GPT OSS 120bRetrieval Type=Hybrid2026.05 | 52 | 66 | |
| RoBERTaLanguage=multilingual, Size=large2026.05 | 52 | 74 | |
| Gemma3 27bRetrieval Type=Dense2026.05 | 50 | 73 | |
| Qwen3 14bRetrieval Type=Dense2026.05 | 49 | 72 | |
| Majority-VoteRetrieval Type=Hybrid2026.05 | 48 | 70 | |
| GPT 5 miniRetrieval Type=Dense2026.05 | 48 | 69 | |
| RoBERTaLanguage=multilingual2026.05 | 48 | 74 | |
| BETOCasing=cased, Language=Spanish2026.05 | 47 | 73 | |
| Majority-VoteRetrieval Type=Dense2026.05 | 46 | 71 | |
| GPT OSS 120bRetrieval Type=Dense2026.05 | 46 | 68 | |
| Gemma3 27bRetrieval Type=Hybrid2026.05 | 46 | 69 | |
| Deepseek-v3.2Retrieval Type=Hybrid2026.05 | 46 | 60 | |
| Meta Llama3.1 8bRetrieval Type=Dense2026.05 | 45 | 56 | |
| Deepseek-v3.2Retrieval Type=Dense2026.05 | 45 | 63 | |
| BETOCasing=uncased, Language=Spanish2026.05 | 45 | 72 | |
| BERT casedCasing=cased2026.05 | 45 | 71 | |
| Longformer2026.05 | 45 | 69 | |
| Qwen3 235bRetrieval Type=Hybrid2026.05 | 44 | 68 | |
| Meta Llama3 70bRetrieval Type=Dense2026.05 | 44 | 64 | |
| GPT 5.2Few-shot shots=32026.05 | 44 | 51 | |
| BERTDomain=legal2026.05 | 44 | 70 | |
| Gemma3 27bFew-shot shots=32026.05 | 43 | 50 | |
| BERTCasing=uncased, Language=multilingual2026.05 | 43 | 71 | |
| BERTCasing=cased, Language=multilingual2026.05 | 43 | 71 | |
| BERT uncasedCasing=uncased2026.05 | 43 | 69 | |
| Qwen3 235bRetrieval Type=Dense2026.05 | 42 | 66 | |
| Gemma3 27bFew-shot shots=52026.05 | 41 | 46 | |
| Qwen3 14bFew-shot shots=32026.05 | 41 | 41 | |
| Meta Llama3.1 8bRetrieval Type=Hybrid2026.05 | 40 | 56 | |
| RoBERTa2026.05 | 40 | 67 | |
| Gemma3 27bFew-shot shots=12026.05 | 38 | 44 | |
| RoBERTalexLanguage=Spanish, Domain=legal2026.05 | 38 | 68 | |
| Qwen3 14bFew-shot shots=52026.05 | 37 | 38 | |
| SVMFeatures=TF-IDF2026.05 | 36 | 64 | |
| GPT 5.2Few-shot shots=52026.05 | 35 | 44 | |
| GPT 5.2Few-shot shots=12026.05 | 34 | 41 | |
| Meta Llama3 70bRetrieval Type=Hybrid2026.05 | 33 | 52 | |
| Meta Llama3.1 8bFew-shot shots=52026.05 | 31 | 34 | |
| Meta Llama3.1 8bFew-shot shots=32026.05 | 27 | 31 | |
| Meta Llama3.1 8bFew-shot shots=12026.05 | 24 | 25 | |
| Qwen3 14bFew-shot shots=12026.05 | 21 | 21 |