Legal Evaluation on LawBench
62AccuracyLawThinker
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| LawThinkerBackbone=Qwen3-32B2026.02 | 62 | — | — | — | |
| PoliLegalLM-30B-A3BParams=30B/A3B, Local=Yes2026.04 | 60.95 | 68.56 | 54 | 67.73 | |
| Qwen3-235B-A22BParams=235B/A22B, Local=Yes2026.04 | 59.74 | 66.72 | 52.38 | 67.2 | |
| GPT-5Local=No2026.04 | 59.16 | 45.68 | 56.89 | 65.38 | |
| GLM-5Params=744B/A30B, Local=Yes2026.04 | 58.74 | 67.47 | 50.33 | 67.07 | |
| Plan-and-SolveBackbone=Qwen3-32B2026.02 | 57.9 | — | — | — | |
| Plan-and-ExecuteBackbone=Qwen3-32B2026.02 | 57 | — | — | — | |
| PoliLegalLM-32BParams=32B, Local=Yes2026.04 | 56.26 | 48.82 | 50.85 | 64.87 | |
| ReActBackbone=Qwen3-32B2026.02 | 56.2 | — | — | — | |
| Search-o1Backbone=Qwen3-32B2026.02 | 55.3 | — | — | — | |
| DeepSeek-V3.2Params=685B/A37B, Local=Yes2026.04 | 54.43 | 64.32 | 44.76 | 64.05 | |
| GPT-42026.06 | 53.85 | — | — | — | |
| Grok-4.2Local=No2026.04 | 53.33 | 51.66 | 45.43 | 63.63 | |
| Doubao-Seed-2.0-ProLocal=No2026.04 | 52.65 | 45.06 | 53.63 | 53.32 | |
| Direct Reasoning2026.02 | 52 | — | — | — | |
| Qwen3.5-397B-A17BParams=397B/A17B, Local=Yes2026.04 | 50.95 | 70.69 | 50.74 | 46.27 | |
| Kimi-K2.5Params=1.1T/A32B, Local=Yes2026.04 | 48.59 | 33.47 | 43.66 | 58.52 | |
| Qwen3-4B-JGARL-Strategy=J, Backbone=Qwen3-4B2026.06 | 48.28 | — | — | — | |
| Qwen3-8B-JGARL-Strategy=J, Backbone=Qwen3-8B2026.06 | 47.99 | — | — | — | |
| Qwen3-8B2026.06 | 47.22 | — | — | — | |
| Qwen3-4B2026.06 | 47.08 | — | — | — | |
| Qwen3-4B-PDGARL-Strategy=PD, Backbone=Qwen3-4B2026.06 | 46.94 | — | — | — | |
| Qwen3-8B-PDGARL-Strategy=PD, Backbone=Qwen3-8B2026.06 | 46.92 | — | — | — | |
| Qwen-Chat-7B2026.06 | 38.99 | — | — | — | |
| ChatLaw-33B2026.06 | 29.14 | — | — | — | |
| Fuzi-Mingcha2026.06 | 28.78 | — | — | — |