Deontic Classification on REGOBLIGATION (test)
92.7F1 ScoreCOMPLIANCENLP
Evaluation results
| Method | Links | |
|---|---|---|
| COMPLIANCENLPRetrieval=With retrieval2026.04 | 92.7 | |
| COMPLIANCENLP w/o MiniCheckRetrieval=With retrieval2026.04 | 92.5 | |
| LLaMA-3-70B*Retrieval=With retrieval2026.04 | 91.8 | |
| COMPLIANCENLP w/o KG rerankingRetrieval=With retrieval2026.04 | 91.2 | |
| GPT-4o + RAGRetrieval=With retrieval2026.04 | 90.5 | |
| COMPLIANCENLP w/o multi-taskRetrieval=With retrieval2026.04 | 90.4 | |
| GPT-4 + RAGRetrieval=With retrieval2026.04 | 89.8 | |
| LLaMA-3-8B + RAGRetrieval=With retrieval2026.04 | 89.8 | |
| GPT-4oRetrieval=Without retrieval, Prompting=5-shot2026.04 | 88.1 | |
| GPT-4Retrieval=Without retrieval, Prompting=5-shot2026.04 | 87.3 | |
| LEGAL-BERTRetrieval=Without retrieval2026.04 | 84.6 | |
| FinBERTRetrieval=Without retrieval2026.04 | 79.2 |