Criminal Court on J1 (eval)
41.5PFS (Procedural Fairness Score)LawThinker
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| LawThinkerMethod Category=Autonomous Tool Usage within Reasoning, Base Model=Qwen3-32B2026.02 | 41.5 | 84.1 | 69.7 | 63.9 | 40.4 | |
| Qwen3-32BMethod Category=Direct Reasoning (Multilingual LLMs)2026.02 | 33.7 | 69.6 | 57.8 | 55.5 | 33.9 | |
| Gemma3-12BMethod Category=Direct Reasoning (Multilingual LLMs)2026.02 | 33.2 | 72.5 | 61.3 | 44.5 | 18.5 | |
| GLM4-9BMethod Category=Direct Reasoning (Multilingual LLMs)2026.02 | 31.4 | 66.7 | 54.4 | 40.3 | 17.5 | |
| Qwen3-14BMethod Category=Direct Reasoning (Multilingual LLMs)2026.02 | 30.7 | 55.1 | 49.6 | 44.8 | 30.8 | |
| Search-o1Method Category=Autonomous Tool Usage within Reasoning, Base Model=Qwen3-32B2026.02 | 28.9 | 84.1 | 67.1 | 60.9 | 39.5 | |
| ReActMethod Category=Workflow-based Methods, Base Model=Qwen3-32B2026.02 | 24.3 | 75.4 | 62.1 | 58.8 | 34.5 | |
| Plan-and-ExecuteMethod Category=Workflow-based Methods, Base Model=Qwen3-32B2026.02 | 24.2 | 75.4 | 57.9 | 58.3 | 34.5 | |
| Plan-and-SolveMethod Category=Workflow-based Methods, Base Model=Qwen3-32B2026.02 | 23.8 | 79.7 | 65.3 | 57.7 | 36.6 | |
| Qwen2.5-7BMethod Category=Direct Reasoning (Multilingual LLMs)2026.02 | 22.4 | 81.2 | 68.1 | 54.8 | 23.9 | |
| Qwen3-8BMethod Category=Direct Reasoning (Multilingual LLMs)2026.02 | 22 | 49.3 | 46.5 | 35.1 | 25.9 | |
| Ministral-8BMethod Category=Direct Reasoning (Multilingual LLMs)2026.02 | 5.7 | 11.6 | 16.2 | 7.4 | 2.7 | |
| ChatLaw2-7BMethod Category=Direct Reasoning (Legal-specific LLMs)2026.02 | 3.7 | 24.6 | 20.9 | 11.9 | 2.2 | |
| LawLLM-7BMethod Category=Direct Reasoning (Legal-specific LLMs)2026.02 | 2.2 | 13 | 7.9 | 10 | 1.6 |