Legal Knowledge Evaluation on CaseHOLD and MMLU-L
79CaseHOLD ScoreGemini-2.5-Pro (25-06-05)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Gemini-2.5-Pro (25-06-05)Params=N/A, Label=false, Knowledge=false2026.01 | 79 | 92.6 | 85.8 | |
| Baseline-w/-labelParams=7B, Label=true, Knowledge=false2026.01 | 76.9 | 78.5 | 77.7 | |
| GPT-4.1 (25-04-14)Params=N/A, Label=false, Knowledge=false2026.01 | 76.4 | 91.7 | 84.1 | |
| Baseline-w/-labelParams=3B, Label=true, Knowledge=false2026.01 | 75.6 | 69.4 | 72.5 | |
| DeepSeek-R1Params=671B, Label=false, Knowledge=false2026.01 | 74.6 | 91.7 | 83.2 | |
| RLAIF (GRPO)Params=7B, Label=false, Knowledge=false2026.01 | 74.4 | 79 | 76.7 | |
| DGRC (Ours)Params=7B, Label=false, Knowledge=false2026.01 | 74.3 | 79.3 | 76.8 | |
| Baseline-w/-labelParams=1.5B, Label=true, Knowledge=false2026.01 | 73.8 | 65.3 | 69.6 | |
| Baseline-w/o-labelParams=7B, Label=false, Knowledge=false2026.01 | 73.6 | 78.5 | 76.1 | |
| DGRC (Ours)Params=3B, Label=false, Knowledge=false2026.01 | 73.1 | 72.7 | 72.9 | |
| RLAIF (GRPO)Params=3B, Label=false, Knowledge=false2026.01 | 73 | 70.8 | 71.9 | |
| Baseline-w/o-labelParams=3B, Label=false, Knowledge=false2026.01 | 72.6 | 69.4 | 71 | |
| DGRC (Ours)Params=1.5B, Label=false, Knowledge=false2026.01 | 71.5 | 67.8 | 69.7 | |
| Qwen2.5-InstructParams=72B, Label=false, Knowledge=false2026.01 | 71.4 | 86.8 | 79.1 | |
| RLAIF (GRPO)Params=1.5B, Label=false, Knowledge=false2026.01 | 71.2 | 64.5 | 67.9 | |
| Baseline-w/o-labelParams=1.5B, Label=false, Knowledge=false2026.01 | 70.3 | 62.8 | 66.6 | |
| Llama3.1-InstructParams=70B, Label=false, Knowledge=false2026.01 | 69.9 | 87.6 | 78.8 | |
| Qwen2.5-InstructParams=32B, Label=false, Knowledge=false2026.01 | 68.2 | 84.3 | 76.3 | |
| Qwen2.5-InstructParams=7B, Label=false, Knowledge=false2026.01 | 63.9 | 75.2 | 69.6 | |
| LegalHalParams=8B, Label=false, Knowledge=true2026.01 | 63.7 | 73.6 | 68.7 | |
| MegaScienceParams=7B, Label=false, Knowledge=true2026.01 | 60.5 | 81 | 70.8 | |
| Llama3.1-InstructParams=8B, Label=false, Knowledge=false2026.01 | 57.7 | 76.9 | 67.3 | |
| MegaScienceParams=3B, Label=false, Knowledge=true2026.01 | 52.4 | 66.1 | 59.3 | |
| Qwen2.5-InstructParams=1.5B, Label=false, Knowledge=false2026.01 | 44.1 | 61.2 | 52.7 | |
| MegaScienceParams=1.5B, Label=false, Knowledge=true2026.01 | 40.9 | 63.6 | 52.3 | |
| Qwen2.5-InstructParams=3B, Label=false, Knowledge=false2026.01 | 38.9 | 61.2 | 50.1 |