Deontic Reasoning on DeonticBench SARA-Num
77.1AccuracyQwen3.5-397B
Evaluation results
| Method | Links | |
|---|---|---|
| Qwen3.5-397BHarness=terminus-kira2026.06 | 77.1 | |
| GPT-5.2Harness=terminus-kira2026.06 | 60 | |
| Qwen3.5-397BHarness=direct2026.06 | 52.8 | |
| Qwen3.5-397BHarness=claude-code2026.06 | 51.4 | |
| GPT-5.2Harness=terminus-22026.06 | 51.4 | |
| Qwen3.5-35BHarness=claude-code2026.06 | 37.1 | |
| Qwen3.5-122BHarness=direct2026.06 | 37 | |
| Qwen3-Coder-480BHarness=claude-code2026.06 | 34.3 | |
| GPT-5.2Harness=codex2026.06 | 34.3 | |
| Qwen3.5-35BHarness=direct2026.06 | 34 | |
| Qwen3-235BHarness=direct2026.06 | 32.1 | |
| GPT-5.2Harness=direct2026.06 | 30.3 | |
| Qwen3.5-122BHarness=claude-code2026.06 | 28.6 | |
| Qwen3.5-397BHarness=terminus-22026.06 | 28.6 | |
| Qwen3-235BHarness=terminus-kira2026.06 | 28.6 | |
| Qwen3-Coder-480BHarness=direct2026.06 | 24.9 | |
| Qwen3.5-122BHarness=codex2026.06 | 22.9 | |
| Qwen3.5-35BHarness=terminus-22026.06 | 22.9 | |
| Kimi-K2Harness=terminus-kira2026.06 | 22.9 | |
| Qwen3.5-122BHarness=terminus-22026.06 | 20 | |
| Qwen3.5-122BHarness=terminus-kira2026.06 | 20 | |
| Kimi-K2Harness=codex2026.06 | 20 | |
| Qwen3-235BHarness=terminus-22026.06 | 17.1 | |
| Qwen3-Coder-480BHarness=terminus-22026.06 | 14.3 | |
| Qwen3-Coder-480BHarness=terminus-kira2026.06 | 14.3 | |
| Qwen3.5-35BHarness=terminus-kira2026.06 | 11.4 | |
| Qwen3-235BHarness=codex2026.06 | 11.4 | |
| Kimi-K2Harness=terminus-22026.06 | 11.4 | |
| Qwen3-Coder-480BHarness=codex2026.06 | 8.6 | |
| Kimi-K2Harness=direct2026.06 | 8.4 |