Deontic Reasoning on DeonticBench Airline (Accuracy)
36.3AccuracyGPT-5.2
Evaluation results
| Method | Links | |
|---|---|---|
| GPT-5.2Harness=terminus-kira2026.06 | 36.3 | |
| Qwen3.5-397BHarness=direct2026.06 | 19.2 | |
| GPT-5.2Harness=terminus-22026.06 | 18.8 | |
| Qwen3.5-122BHarness=direct2026.06 | 15 | |
| Qwen3.5-35BHarness=direct2026.06 | 13.7 | |
| Qwen3-235BHarness=direct2026.06 | 12.8 | |
| Qwen3.5-122BHarness=claude-code2026.06 | 11.3 | |
| Qwen3.5-397BHarness=claude-code2026.06 | 10 | |
| Kimi-K2Harness=direct2026.06 | 9 | |
| Qwen3.5-35BHarness=claude-code2026.06 | 8.8 | |
| Qwen3-Coder-480BHarness=claude-code2026.06 | 5 | |
| Qwen3.5-122BHarness=terminus-kira2026.06 | 5 | |
| Kimi-K2Harness=terminus-kira2026.06 | 5 | |
| Qwen3.5-122BHarness=terminus-22026.06 | 3.8 | |
| Qwen3-235BHarness=terminus-kira2026.06 | 3.8 | |
| GPT-5.2Harness=direct2026.06 | 2.5 | |
| Qwen3-Coder-480BHarness=direct2026.06 | 2.1 | |
| Qwen3-Coder-480BHarness=terminus-kira2026.06 | 1.3 | |
| Qwen3.5-122BHarness=codex2026.06 | 1.3 | |
| Qwen3.5-35BHarness=terminus-22026.06 | 1.3 | |
| Qwen3.5-35BHarness=terminus-kira2026.06 | 1.3 | |
| Qwen3.5-397BHarness=terminus-22026.06 | 1.3 | |
| Qwen3-235BHarness=terminus-22026.06 | 1.3 | |
| Qwen3-Coder-480BHarness=codex2026.06 | 0 | |
| Qwen3-Coder-480BHarness=terminus-22026.06 | 0 | |
| Qwen3.5-397BHarness=terminus-kira2026.06 | 0 | |
| Qwen3-235BHarness=codex2026.06 | 0 | |
| Kimi-K2Harness=codex2026.06 | 0 | |
| Kimi-K2Harness=terminus-22026.06 | 0 | |
| GPT-5.2Harness=codex2026.06 | 0 |