Entailment on Privacy Policy Entailment (Priv. Ent.)
85AccuracyLegalDrill-1.7B
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| LegalDrill-1.7BEvaluation Protocol=Distillation, Teacher Model=Qwen3-30B-A3B-Instruct2026.04 | 85 | 80 | 59 | |
| Qwen3-30B-A3B-InstructEvaluation Protocol=Zero Shot2026.04 | 83 | 75 | 65 | |
| LegalDrill-0.6BEvaluation Protocol=Distillation, Teacher Model=Qwen3-30B-A3B-Instruct2026.04 | 81 | 80 | 58 | |
| DeepSeek ESFT-16BEvaluation Protocol=Zero Shot2026.04 | 75 | 64 | 73 | |
| GPT-4oEvaluation Protocol=Zero Shot2026.04 | 67 | 30 | 60 | |
| Law-LLM-13BEvaluation Protocol=Zero Shot2026.04 | 64 | 38 | 29 | |
| LegalDrill-1.7BEvaluation Protocol=Distillation, Teacher Model=GPT-4o2026.04 | 60 | 31 | 52 | |
| LegalDrill-0.6BEvaluation Protocol=Distillation, Teacher Model=GPT-4o2026.04 | 59 | 32 | 52 | |
| Qwen3-1.7BEvaluation Protocol=Zero Shot2026.04 | 47 | 45 | 30 | |
| Qwen3-0.6BEvaluation Protocol=Zero Shot2026.04 | 30 | 29 | 24 | |
| DiscLaw-13BEvaluation Protocol=Zero Shot2026.04 | 1 | 1 | 1 |