Legal case grading agreement on Criminal law
0.599QWKGPT-5†
Evaluation results
| Method | Links | |
|---|---|---|
| GPT-5†Reasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.599 | |
| Gemini-2.5-ProReasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.565 | |
| GPT-5†Reasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.557 | |
| GPT-5-miniReasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.499 | |
| GPT-4.1Reasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.478 | |
| Gemini-2.5-ProReasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.475 | |
| GPT-5.2Reasoning Mode=Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.465 | |
| Qwen3-235B-Th.Reasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.453 | |
| GPT-5.2Reasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.431 | |
| Gemini-2.5-ProReasoning Mode=Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.43 | |
| GPT-5-miniReasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.425 | |
| GPT-4.1Reasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.405 | |
| GPT-5†Reasoning Mode=Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.392 | |
| GPT-OSS-120BReasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.353 | |
| Mistral-Large-3-675BReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.347 | |
| Qwen3-Next-80B-Th.Reasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.336 | |
| GPT-5.2Reasoning Mode=Reasoning, Prompting Strategy=Task Agnostic2026.05 | 0.311 | |
| GPT-5†Reasoning Mode=Reasoning, Prompting Strategy=Task Agnostic2026.05 | 0.309 | |
| Qwen3-Next-80B-Th.Reasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.309 | |
| GPT-OSS-120BReasoning Mode=Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.307 | |
| Qwen3-30B-Th.Reasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.306 | |
| Qwen3-235B-Th.Reasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.305 | |
| GPT-5.2Reasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.298 | |
| DeepSeek-685B-V3.1Reasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.287 | |
| GPT-OSS-120BReasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.284 | |
| DeepSeek-685B-V3.1Reasoning Mode=Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.283 | |
| GPT-OSS-20BReasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.282 | |
| DeepSeek-685B-V3.1Reasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.277 | |
| GPT-5-miniReasoning Mode=Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.27 | |
| Mistral-Large-3-675BReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.27 | |
| Qwen3-Next-80B-Th.Reasoning Mode=Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.269 | |
| GPT-OSS-20BReasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.25 | |
| GPT-4.1Reasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.236 | |
| Qwen3-235B-Th.Reasoning Mode=Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.234 | |
| Mistral-Large-3-675BReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.205 | |
| GPT-5.1Reasoning Mode=Reasoning, Prompting Strategy=Task Agnostic2026.05 | 0.181 | |
| GPT-OSS-20BReasoning Mode=Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.171 | |
| GPT-4oReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.168 | |
| Gemini-2.5-ProReasoning Mode=Reasoning, Prompting Strategy=Task Agnostic2026.05 | 0.161 | |
| GPT-4.1-miniReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.156 | |
| GPT-5.1Reasoning Mode=Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.151 | |
| GPT-4.1-miniReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.149 | |
| GPT-5.1Reasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.145 | |
| GPT-4oReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.145 | |
| Qwen3-30B-Th.Reasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.129 | |
| Gemma-3-27B-itReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.124 | |
| Qwen3-32BReasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.117 | |
| Apertus-70B-itReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.117 | |
| GPT-4oReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.114 | |
| Ministral-3-14B-itReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.113 | |
| GPT-4o-miniReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.107 | |
| GPT-4.1-miniReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.103 | |
| Qwen3-30B-Th.Reasoning Mode=Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.101 | |
| GPT-5-miniReasoning Mode=Reasoning, Prompting Strategy=Task Agnostic2026.05 | 0.097 | |
| Qwen3-235BReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.096 | |
| DeepSeek-685B-V3.1Reasoning Mode=Reasoning, Prompting Strategy=Task Agnostic2026.05 | 0.088 | |
| GPT-4o-miniReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.087 | |
| GPT-5.1Reasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.086 | |
| QwQ-32BReasoning Mode=Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.083 | |
| Qwen3-30B-it.Reasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.082 | |
| Gemma-3-27B-itReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.075 | |
| Llama-3.3-70B-itReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.074 | |
| Ministral-3-14B-Rea.Reasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.072 | |
| QwQ-32BReasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.068 | |
| Mistral-Large-3-675BReasoning Mode=Non-Reasoning, Prompting Strategy=Task Agnostic2026.05 | 0.067 | |
| GPT-OSS-20BReasoning Mode=Reasoning, Prompting Strategy=Task Agnostic2026.05 | 0.065 | |
| Qwen3-32BReasoning Mode=Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.065 | |
| Ministral-3-14B-itReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.064 | |
| GPT-4oReasoning Mode=Non-Reasoning, Prompting Strategy=Task Agnostic2026.05 | 0.063 | |
| Llama-3.3-70B-itReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.062 | |
| Ministral-3-14B-itReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.058 | |
| Apertus-70B-itReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.058 | |
| Qwen3-Next-80B-itReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.056 | |
| Qwen3-235BReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.054 | |
| Ministral-3-14B-Rea.Reasoning Mode=Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.052 | |
| GPT-4.1Reasoning Mode=Non-Reasoning, Prompting Strategy=Task Agnostic2026.05 | 0.052 | |
| Qwen3-Next-80B-itReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.052 | |
| Qwen3-30B-it.Reasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.052 | |
| Qwen3-Next-80B-Th.Reasoning Mode=Reasoning, Prompting Strategy=Task Agnostic2026.05 | 0.05 | |
| QwQ-32BReasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.05 | |
| Qwen3-32BReasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.042 | |
| EuroLLM-22B-itReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.037 | |
| Qwen3-30B-it.Reasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.036 | |
| Ministral-3-14B-Rea.Reasoning Mode=Reasoning, Prompting Strategy=Task Agnostic2026.05 | 0.03 | |
| Qwen3-Next-80B-itReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.03 | |
| Llama-3.3-70B-itReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.027 | |
| Ministral-3-14B-itReasoning Mode=Non-Reasoning, Prompting Strategy=Task Agnostic2026.05 | 0.022 | |
| Gemma-3-27B-itReasoning Mode=Non-Reasoning, Prompting Strategy=Task Agnostic2026.05 | 0.015 | |
| GPT-4.1-miniReasoning Mode=Non-Reasoning, Prompting Strategy=Task Agnostic2026.05 | 0.013 | |
| GPT-4o-miniReasoning Mode=Non-Reasoning, Prompting Strategy=Task Agnostic2026.05 | 0.013 | |
| Qwen3-235BReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.012 | |
| Gemma-3-27B-itReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.011 | |
| Qwen3-Next-80B-itReasoning Mode=Non-Reasoning, Prompting Strategy=Task Agnostic2026.05 | 0.01 | |
| Qwen3-235B-Th.Reasoning Mode=Reasoning, Prompting Strategy=Task Agnostic2026.05 | 0.008 | |
| Qwen3-235BReasoning Mode=Non-Reasoning, Prompting Strategy=Task Agnostic2026.05 | 0.007 | |
| Qwen3-30B-it.Reasoning Mode=Non-Reasoning, Prompting Strategy=Task Agnostic2026.05 | 0.006 | |
| Apertus-70B-itReasoning Mode=Non-Reasoning, Prompting Strategy=Task Agnostic2026.05 | 0.005 | |
| Llama-3.3-70B-itReasoning Mode=Non-Reasoning, Prompting Strategy=Task Agnostic2026.05 | 0.004 | |
| Apertus-70B-itReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.002 | |
| Qwen3-32BReasoning Mode=Reasoning, Prompting Strategy=Task Agnostic2026.05 | 0 |