Legal Consultation on J1-EVAL
52.4Average ScoreLawThinker
Evaluation results
| Method | Links | |
|---|---|---|
| LawThinkerMethod Category=Autonomous Tool Usage within Reasoning, Base Model=Qwen3-32B2026.02 | 52.4 | |
| Search-o1Method Category=Autonomous Tool Usage within Reasoning, Base Model=Qwen3-32B2026.02 | 48.4 | |
| Plan-and-ExecuteMethod Category=Workflow-based Methods, Base Model=Qwen3-32B2026.02 | 47.9 | |
| Qwen3-32BMethod Category=Direct Reasoning (Multilingual LLMs)2026.02 | 47.4 | |
| ReActMethod Category=Workflow-based Methods, Base Model=Qwen3-32B2026.02 | 46.8 | |
| Plan-and-SolveMethod Category=Workflow-based Methods, Base Model=Qwen3-32B2026.02 | 46.4 | |
| Qwen3-14BMethod Category=Direct Reasoning (Multilingual LLMs)2026.02 | 43.3 | |
| Qwen3-8BMethod Category=Direct Reasoning (Multilingual LLMs)2026.02 | 40.3 | |
| Qwen2.5-7BMethod Category=Direct Reasoning (Multilingual LLMs)2026.02 | 40.1 | |
| ChatLaw2-7BMethod Category=Direct Reasoning (Legal-specific LLMs)2026.02 | 36.5 | |
| LawLLM-7BMethod Category=Direct Reasoning (Legal-specific LLMs)2026.02 | 35.3 | |
| GLM4-9BMethod Category=Direct Reasoning (Multilingual LLMs)2026.02 | 34.2 | |
| Gemma3-12BMethod Category=Direct Reasoning (Multilingual LLMs)2026.02 | 33.5 | |
| Ministral-8BMethod Category=Direct Reasoning (Multilingual LLMs)2026.02 | 32.1 |