Knowledge Questioning on J1 EVAL
64.1Average ScoreLawThinker
Evaluation results
| Method | Links | |
|---|---|---|
| LawThinkerMethod Category=Autonomous Tool Usage within Reasoning, Base Model=Qwen3-32B2026.02 | 64.1 | |
| Search-o1Method Category=Autonomous Tool Usage within Reasoning, Base Model=Qwen3-32B2026.02 | 63.5 | |
| Plan-and-ExecuteMethod Category=Workflow-based Methods, Base Model=Qwen3-32B2026.02 | 61.9 | |
| ReActMethod Category=Workflow-based Methods, Base Model=Qwen3-32B2026.02 | 61.3 | |
| Plan-and-SolveMethod Category=Workflow-based Methods, Base Model=Qwen3-32B2026.02 | 60.9 | |
| Qwen3-32BMethod Category=Direct Reasoning (Multilingual LLMs)2026.02 | 59.6 | |
| Qwen3-14BMethod Category=Direct Reasoning (Multilingual LLMs)2026.02 | 57.7 | |
| GLM4-9BMethod Category=Direct Reasoning (Multilingual LLMs)2026.02 | 55.6 | |
| Qwen2.5-7BMethod Category=Direct Reasoning (Multilingual LLMs)2026.02 | 52.9 | |
| ChatLaw2-7BMethod Category=Direct Reasoning (Legal-specific LLMs)2026.02 | 52.9 | |
| Qwen3-8BMethod Category=Direct Reasoning (Multilingual LLMs)2026.02 | 52.8 | |
| Gemma3-12BMethod Category=Direct Reasoning (Multilingual LLMs)2026.02 | 51.2 | |
| Ministral-8BMethod Category=Direct Reasoning (Multilingual LLMs)2026.02 | 49.1 | |
| LawLLM-7BMethod Category=Direct Reasoning (Legal-specific LLMs)2026.02 | 48.7 |