Legal case grading agreement on Public law
0.911QWKGPT-5†
Evaluation results
| Method | Links | |
|---|---|---|
| GPT-5†Reasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.911 | |
| Gemini-2.5-ProReasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.884 | |
| GPT-5.2Reasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.873 | |
| Gemini-2.5-ProReasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.86 | |
| GPT-5†Reasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.846 | |
| GPT-5†Reasoning Mode=Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.793 | |
| Mistral-Large-3-675BReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.79 | |
| Qwen3-Next-80B-Th.Reasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.789 | |
| Qwen3-235B-Th.Reasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.787 | |
| DeepSeek-685B-V3.1Reasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.785 | |
| GPT-OSS-120BReasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.744 | |
| GPT-OSS-20BReasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.712 | |
| Mistral-Large-3-675BReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.709 | |
| GPT-5-miniReasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.697 | |
| GPT-5.2Reasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.685 | |
| GPT-4.1Reasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.678 | |
| Gemini-2.5-ProReasoning Mode=Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.676 | |
| GPT-4.1Reasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.659 | |
| GPT-5-miniReasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.641 | |
| Qwen3-Next-80B-Th.Reasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.635 | |
| GPT-OSS-20BReasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.628 | |
| GPT-5.2Reasoning Mode=Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.621 | |
| Qwen3-235B-Th.Reasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.604 | |
| DeepSeek-685B-V3.1Reasoning Mode=Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.6 | |
| GPT-4.1-miniReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.598 | |
| GPT-5.1Reasoning Mode=Reasoning, Prompting Strategy=Task Agnostic2026.05 | 0.596 | |
| GPT-4.1Reasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.589 | |
| DeepSeek-685B-V3.1Reasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.581 | |
| GPT-5†Reasoning Mode=Reasoning, Prompting Strategy=Task Agnostic2026.05 | 0.576 | |
| Gemini-2.5-ProReasoning Mode=Reasoning, Prompting Strategy=Task Agnostic2026.05 | 0.568 | |
| GPT-5-miniReasoning Mode=Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.558 | |
| GPT-OSS-20BReasoning Mode=Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.538 | |
| Ministral-3-14B-itReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.538 | |
| Qwen3-Next-80B-Th.Reasoning Mode=Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.527 | |
| Ministral-3-14B-itReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.514 | |
| Qwen3-30B-Th.Reasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.506 | |
| Qwen3-30B-Th.Reasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.505 | |
| DeepSeek-685B-V3.1Reasoning Mode=Reasoning, Prompting Strategy=Task Agnostic2026.05 | 0.501 | |
| GPT-5.1Reasoning Mode=Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.47 | |
| GPT-OSS-120BReasoning Mode=Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.469 | |
| GPT-4oReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.466 | |
| Qwen3-30B-Th.Reasoning Mode=Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.461 | |
| Qwen3-235BReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.456 | |
| Qwen3-235B-Th.Reasoning Mode=Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.45 | |
| Mistral-Large-3-675BReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.423 | |
| GPT-4oReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.409 | |
| GPT-OSS-120BReasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.4 | |
| GPT-5.1Reasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.399 | |
| Ministral-3-14B-itReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.375 | |
| QwQ-32BReasoning Mode=Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.367 | |
| Qwen3-235BReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.366 | |
| GPT-5.1Reasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.361 | |
| GPT-5.2Reasoning Mode=Reasoning, Prompting Strategy=Task Agnostic2026.05 | 0.356 | |
| QwQ-32BReasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.343 | |
| GPT-4.1Reasoning Mode=Non-Reasoning, Prompting Strategy=Task Agnostic2026.05 | 0.338 | |
| GPT-4.1-miniReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.321 | |
| Apertus-70B-itReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.314 | |
| Ministral-3-14B-Rea.Reasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.287 | |
| GPT-4oReasoning Mode=Non-Reasoning, Prompting Strategy=Task Agnostic2026.05 | 0.283 | |
| GPT-4oReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.279 | |
| Qwen3-Next-80B-itReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.263 | |
| Qwen3-Next-80B-itReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.262 | |
| GPT-5-miniReasoning Mode=Reasoning, Prompting Strategy=Task Agnostic2026.05 | 0.224 | |
| Qwen3-235BReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.206 | |
| Qwen3-Next-80B-itReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.2 | |
| GPT-4.1-miniReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.176 | |
| Llama-3.3-70B-itReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.175 | |
| Llama-3.3-70B-itReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.172 | |
| Qwen3-32BReasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.156 | |
| Qwen3-235B-Th.Reasoning Mode=Reasoning, Prompting Strategy=Task Agnostic2026.05 | 0.153 | |
| QwQ-32BReasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.146 | |
| Qwen3-32BReasoning Mode=Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.136 | |
| Qwen3-32BReasoning Mode=Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.122 | |
| Llama-3.3-70B-itReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.118 | |
| Mistral-Large-3-675BReasoning Mode=Non-Reasoning, Prompting Strategy=Task Agnostic2026.05 | 0.113 | |
| GPT-4o-miniReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.113 | |
| Qwen3-30B-it.Reasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.104 | |
| GPT-4.1-miniReasoning Mode=Non-Reasoning, Prompting Strategy=Task Agnostic2026.05 | 0.096 | |
| Ministral-3-14B-Rea.Reasoning Mode=Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.095 | |
| Qwen3-30B-it.Reasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.092 | |
| QwQ-32BReasoning Mode=Reasoning, Prompting Strategy=Task Agnostic2026.05 | 0.08 | |
| Qwen3-Next-80B-itReasoning Mode=Non-Reasoning, Prompting Strategy=Task Agnostic2026.05 | 0.074 | |
| GPT-OSS-20BReasoning Mode=Reasoning, Prompting Strategy=Task Agnostic2026.05 | 0.073 | |
| EuroLLM-22B-itReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.066 | |
| Gemma-3-27B-itReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.06 | |
| Gemma-3-27B-itReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.053 | |
| Qwen3-30B-it.Reasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.052 | |
| Gemma-3-27B-itReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Rubric+Sol2026.05 | 0.044 | |
| GPT-4o-miniReasoning Mode=Non-Reasoning, Prompting Strategy=Task Agnostic2026.05 | 0.037 | |
| Apertus-70B-itReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.037 | |
| Ministral-3-14B-Rea.Reasoning Mode=Reasoning, Prompting Strategy=Task Agnostic2026.05 | 0.036 | |
| GPT-4o-miniReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Solution2026.05 | 0.034 | |
| Qwen3-235BReasoning Mode=Non-Reasoning, Prompting Strategy=Task Agnostic2026.05 | 0.026 | |
| EuroLLM-22B-itReasoning Mode=Non-Reasoning, Prompting Strategy=Task Agnostic2026.05 | 0.019 | |
| GPT-4o-miniReasoning Mode=Non-Reasoning, Prompting Strategy=Instr.+Rubric2026.05 | 0.017 | |
| Ministral-3-14B-itReasoning Mode=Non-Reasoning, Prompting Strategy=Task Agnostic2026.05 | 0.013 | |
| Gemma-3-27B-itReasoning Mode=Non-Reasoning, Prompting Strategy=Task Agnostic2026.05 | 0.011 | |
| Qwen3-32BReasoning Mode=Reasoning, Prompting Strategy=Task Agnostic2026.05 | 0.006 | |
| Llama-3.3-70B-itReasoning Mode=Non-Reasoning, Prompting Strategy=Task Agnostic2026.05 | 0.005 | |
| Qwen3-30B-it.Reasoning Mode=Non-Reasoning, Prompting Strategy=Task Agnostic2026.05 | 0.001 |