Defence Drafting on J1 EVAL
81.4FORLawThinker
Evaluation results
| Method | Links | ||
|---|---|---|---|
| LawThinkerMethod Category=Autonomous Tool Usage within Reasoning, Base Model=Qwen3-32B2026.02 | 81.4 | 60.3 | |
| GLM4-9BMethod Category=Direct Reasoning (Multilingual LLMs)2026.02 | 57 | 54.8 | |
| Plan-and-SolveMethod Category=Workflow-based Methods, Base Model=Qwen3-32B2026.02 | 42.7 | 59 | |
| Search-o1Method Category=Autonomous Tool Usage within Reasoning, Base Model=Qwen3-32B2026.02 | 42.7 | 54.9 | |
| ReActMethod Category=Workflow-based Methods, Base Model=Qwen3-32B2026.02 | 41.2 | 54.5 | |
| Ministral-8BMethod Category=Direct Reasoning (Multilingual LLMs)2026.02 | 40.1 | 52.8 | |
| Plan-and-ExecuteMethod Category=Workflow-based Methods, Base Model=Qwen3-32B2026.02 | 40.1 | 55.4 | |
| Qwen3-14BMethod Category=Direct Reasoning (Multilingual LLMs)2026.02 | 32.6 | 59 | |
| Qwen3-32BMethod Category=Direct Reasoning (Multilingual LLMs)2026.02 | 30.8 | 51 | |
| Qwen2.5-7BMethod Category=Direct Reasoning (Multilingual LLMs)2026.02 | 26.5 | 50.4 | |
| LawLLM-7BMethod Category=Direct Reasoning (Legal-specific LLMs)2026.02 | 26.5 | 43.6 | |
| ChatLaw2-7BMethod Category=Direct Reasoning (Legal-specific LLMs)2026.02 | 15.8 | 18.9 | |
| Qwen3-8BMethod Category=Direct Reasoning (Multilingual LLMs)2026.02 | 12.2 | 40.7 | |
| Gemma3-12BMethod Category=Direct Reasoning (Multilingual LLMs)2026.02 | 11.8 | 51.6 |