Legal Research and Reasoning on LegalSearch In-domain
96.73LAR (Rouge-L)LegalSearch-R1
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| LegalSearch-R1Think=true, Web=true, RAG=true2026.05 | 96.73 | 35.16 | 50 | 87.5 | 62.5 | 13.28 | 46.09 | 55.9 | |
| DeepSeek-V3.2Think=true, Web=false, RAG=false2026.05 | 63.3 | 42.97 | 62.5 | 78.12 | 43.75 | 13.28 | 50 | 50.56 | |
| DeepPlannerThink=true, Web=true, RAG=false2026.05 | 54.02 | 20.31 | 39.84 | 75 | 36.72 | 13.28 | 35.16 | 39.19 | |
| LegalDeltaThink=true, Web=false, RAG=false2026.05 | 52.31 | 26.17 | 45.31 | 81.25 | 60.16 | 17.97 | 54.69 | 48.27 | |
| LegalDeltaThink=false, Web=false, RAG=false2026.05 | 52.3 | 24.61 | 47.66 | 81.25 | 60.94 | 20.31 | 53.91 | 48.71 | |
| Qwen-3-30b-A3bThink=false, Web=false, RAG=false2026.05 | 51.27 | 41.41 | 55.47 | 85.16 | 42.19 | 10.16 | 48.44 | 47.73 | |
| Qwen-3-30b-A3bThink=true, Web=false, RAG=false2026.05 | 43.14 | 37.11 | 50.78 | 82.81 | 46.09 | 12.5 | 53.91 | 46.62 | |
| DeepResearcherThink=true, Web=true, RAG=false2026.05 | 40.51 | 18.75 | 39.06 | 62.5 | 38.28 | 13.28 | 36.72 | 35.59 | |
| QwQ-32bThink=true, Web=false, RAG=false2026.05 | 39.64 | 38.67 | 55.47 | 53.91 | 37.5 | 10.16 | 53.91 | 41.32 | |
| Qwen-3-14bThink=false, Web=false, RAG=false2026.05 | 38.74 | 33.59 | 52.34 | 82.81 | 46.88 | 8.59 | 50.78 | 44.82 | |
| DISC-LawLLMThink=false, Web=false, RAG=false2026.05 | 37.41 | 7.81 | 14.84 | 46.88 | 54.69 | 7.81 | 27.34 | 28.11 | |
| Qwen-2.5-7bThink=false, Web=false, RAG=false2026.05 | 36.39 | 14.84 | 35.94 | 68.75 | 40.62 | 9.38 | 31.25 | 33.88 | |
| DISC-LawLLMThink=true, Web=false, RAG=false2026.05 | 35.37 | 8.59 | 25 | 39.84 | 53.12 | 0 | 19.53 | 25.92 | |
| Search-r1-instructThink=true, Web=false, RAG=true2026.05 | 35.31 | 19.53 | 32.81 | 65.62 | 34.38 | 14.84 | 39.06 | 34.51 | |
| Qwen-3-8bThink=false, Web=false, RAG=false2026.05 | 33.53 | 28.52 | 45.31 | 78.12 | 37.5 | 17.19 | 46.88 | 41.01 | |
| Search-r1-baseThink=true, Web=false, RAG=true2026.05 | 21.38 | 16.02 | 43.75 | 64.06 | 22.66 | 3.91 | 36.72 | 29.78 | |
| o4-miniThink=true, Web=false, RAG=false2026.05 | 19.92 | 18.75 | 37.5 | 57.03 | 34.38 | 10.16 | 33.59 | 30.19 | |
| GPT-4o-miniThink=false, Web=false, RAG=false2026.05 | 17.15 | 7.42 | 29.69 | 13.28 | 19.53 | 10.94 | 16.41 | 16.35 |