Legal Examination on legal examination tasks Out-of-domain
79.69CPADeepSeek-V3.2
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| DeepSeek-V3.2Think=true, Web=false, RAG=false2026.05 | 79.69 | 42.19 | 42.97 | 65.62 | 80.47 | 87.5 | 66.41 | |
| Qwen-3-30b-A3bcategory=Vanilla LLMs, Think=false, Web=false, RAG=false2026.05 | 76.56 | 43.75 | 40.62 | 64.84 | 72.66 | 84.38 | 63.8 | |
| Qwen-3-30b-A3bcategory=LRMs, Think=false, Web=false, RAG=false2026.05 | 75.78 | 44.53 | 42.19 | 71.09 | 78.12 | 85.94 | 66.28 | |
| QwQ-32bThink=false, Web=false, RAG=false2026.05 | 75 | 33.59 | 38.28 | 53.91 | 60.94 | 81.25 | 57.16 | |
| Qwen-3-14bThink=false, Web=false, RAG=false2026.05 | 71.09 | 42.97 | 41.41 | 61.72 | 76.56 | 84.38 | 63.02 | |
| LegalSearch-R1Think=true, Web=true, RAG=true2026.05 | 70.31 | 42.19 | 41.41 | 70.31 | 74.22 | 83.59 | 63.67 | |
| DeepPlannerThink=true, Web=true, RAG=false2026.05 | 63.28 | 27.34 | 25.78 | 59.38 | 67.19 | 80.47 | 53.91 | |
| DISC-LawLLMThink=false, Web=false, RAG=false2026.05 | 59.38 | 1.56 | 3.91 | 11.72 | 39.84 | 41.41 | 19.01 | |
| LegalDeltaThink=true, Web=false, RAG=false2026.05 | 59.38 | 36.72 | 34.38 | 57.81 | 52.34 | 55.47 | 49.35 | |
| Qwen-2.5-7bThink=false, Web=false, RAG=false2026.05 | 58.59 | 26.56 | 28.12 | 53.12 | 65.62 | 65.62 | 49.61 | |
| Qwen-3-8bThink=false, Web=false, RAG=false2026.05 | 58.59 | 32.03 | 39.06 | 65.62 | 75 | 79.69 | 58.33 | |
| Search-r1-baseThink=true, Web=false, RAG=true2026.05 | 57.03 | 28.12 | 24.22 | 60.16 | 65.62 | 78.12 | 52.21 | |
| LegalDeltaThink=false, Web=false, RAG=false2026.05 | 56.25 | 40.62 | 34.38 | 53.12 | 45.31 | 50.78 | 46.74 | |
| DeepResearcherThink=true, Web=true, RAG=false2026.05 | 52.34 | 23.43 | 27.34 | 54.69 | 60.94 | 71.88 | 48.43 | |
| Search-r1-instructThink=true, Web=false, RAG=true2026.05 | 50 | 25 | 28.91 | 51.56 | 61.72 | 69.53 | 47.79 | |
| o4-miniThink=true, Web=false, RAG=false2026.05 | 48.44 | 36.72 | 28.91 | 51.56 | 63.28 | 66.41 | 49.22 | |
| DISC-LawLLMThink=true, Web=false, RAG=false2026.05 | 39.06 | 10.94 | 7.03 | 35.94 | 51.56 | 64.06 | 34.77 | |
| GPT-4o-miniThink=false, Web=false, RAG=false2026.05 | 28.91 | 22.66 | 23.44 | 35.94 | 41.41 | 44.53 | 32.81 |