Civil Court on J1 EVAL
50.3PFS ScoreLawThinker
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| LawThinkerMethod Category=Autonomous Tool Usage within Reasoning, Base Model=Qwen3-32B2026.02 | 50.3 | 28.5 | 44 | 23.2 | |
| Gemma3-12BMethod Category=Direct Reasoning (Multilingual LLMs)2026.02 | 42.7 | 17.7 | 32.8 | 14.8 | |
| Qwen3-32BMethod Category=Direct Reasoning (Multilingual LLMs)2026.02 | 42.5 | 16.7 | 35.7 | 17.3 | |
| Search-o1Method Category=Autonomous Tool Usage within Reasoning, Base Model=Qwen3-32B2026.02 | 40.6 | 20.5 | 36.3 | 21.2 | |
| Plan-and-ExecuteMethod Category=Workflow-based Methods, Base Model=Qwen3-32B2026.02 | 40.5 | 23.8 | 42.6 | 20.5 | |
| ReActMethod Category=Workflow-based Methods, Base Model=Qwen3-32B2026.02 | 39.2 | 21.6 | 38.3 | 20.8 | |
| Plan-and-SolveMethod Category=Workflow-based Methods, Base Model=Qwen3-32B2026.02 | 38 | 23.4 | 38.7 | 19.4 | |
| Qwen2.5-7BMethod Category=Direct Reasoning (Multilingual LLMs)2026.02 | 23.4 | 9.1 | 20 | 8.5 | |
| GLM4-9BMethod Category=Direct Reasoning (Multilingual LLMs)2026.02 | 20.8 | 5.9 | 16.5 | 4.7 | |
| Ministral-8BMethod Category=Direct Reasoning (Multilingual LLMs)2026.02 | 15.1 | 5.7 | 10.1 | 4.3 | |
| Qwen3-8BMethod Category=Direct Reasoning (Multilingual LLMs)2026.02 | 12.9 | 9.9 | 23.1 | 14.7 | |
| Qwen3-14BMethod Category=Direct Reasoning (Multilingual LLMs)2026.02 | 6.8 | 4.1 | 5.9 | 3.9 | |
| ChatLaw2-7BMethod Category=Direct Reasoning (Legal-specific LLMs)2026.02 | 3.7 | 5.6 | 7.8 | — | |
| LawLLM-7BMethod Category=Direct Reasoning (Legal-specific LLMs)2026.02 | 3.4 | — | — | — |