# Deontic Reasoning on DeonticBench Airline (Accuracy) Source: [Deontic Reasoning on DeonticBench Airline (Accuracy)]() Collection index: [llms.txt]() Data updated: 2026-06-04T13:12:01.623Z Evaluation of model performance on Deontic Reasoning tasks using the DeonticBench Airline dataset, measured by Accuracy. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Deontic Reasoning | | Dataset (including any stated split) | DeonticBench Airline | | Metric | Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 30 of 30 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Accuracy](): Higher is better; unit: PERCENT. ## Recorded results | Method | Accuracy | Result date | Report paper | | --- | --- | --- | --- | | GPT-5.2 | 36.3 | Not recorded | 6a2106eac10484bac23f0605 | | Qwen3.5-397B | 19.2 | Not recorded | 6a2106eac10484bac23f0605 | | GPT-5.2 | 18.8 | Not recorded | 6a2106eac10484bac23f0605 | | Qwen3.5-122B | 15 | Not recorded | 6a2106eac10484bac23f0605 | | Qwen3.5-35B | 13.7 | Not recorded | 6a2106eac10484bac23f0605 | | Qwen3-235B | 12.8 | Not recorded | 6a2106eac10484bac23f0605 | | Qwen3.5-122B | 11.3 | Not recorded | 6a2106eac10484bac23f0605 | | Qwen3.5-397B | 10 | Not recorded | 6a2106eac10484bac23f0605 | | Kimi-K2 | 9 | Not recorded | 6a2106eac10484bac23f0605 | | Qwen3.5-35B | 8.8 | Not recorded | 6a2106eac10484bac23f0605 | | Qwen3-Coder-480B | 5 | Not recorded | 6a2106eac10484bac23f0605 | | Qwen3.5-122B | 5 | Not recorded | 6a2106eac10484bac23f0605 | | Kimi-K2 | 5 | Not recorded | 6a2106eac10484bac23f0605 | | Qwen3.5-122B | 3.8 | Not recorded | 6a2106eac10484bac23f0605 | | Qwen3-235B | 3.8 | Not recorded | 6a2106eac10484bac23f0605 | | GPT-5.2 | 2.5 | Not recorded | 6a2106eac10484bac23f0605 | | Qwen3-Coder-480B | 2.1 | Not recorded | 6a2106eac10484bac23f0605 | | Qwen3-Coder-480B | 1.3 | Not recorded | 6a2106eac10484bac23f0605 | | Qwen3.5-122B | 1.3 | Not recorded | 6a2106eac10484bac23f0605 | | Qwen3.5-35B | 1.3 | Not recorded | 6a2106eac10484bac23f0605 | | Qwen3.5-35B | 1.3 | Not recorded | 6a2106eac10484bac23f0605 | | Qwen3.5-397B | 1.3 | Not recorded | 6a2106eac10484bac23f0605 | | Qwen3-235B | 1.3 | Not recorded | 6a2106eac10484bac23f0605 | | Qwen3-Coder-480B | 0 | Not recorded | 6a2106eac10484bac23f0605 | | Qwen3-Coder-480B | 0 | Not recorded | 6a2106eac10484bac23f0605 | | Qwen3.5-397B | 0 | Not recorded | 6a2106eac10484bac23f0605 | | Qwen3-235B | 0 | Not recorded | 6a2106eac10484bac23f0605 | | Kimi-K2 | 0 | Not recorded | 6a2106eac10484bac23f0605 | | Kimi-K2 | 0 | Not recorded | 6a2106eac10484bac23f0605 | | GPT-5.2 | 0 | Not recorded | 6a2106eac10484bac23f0605 | ### GPT-5.2 Accuracy: 36.3. Result ID: 6a21799efbc373592c1ee465. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Harness":"terminus-kira"}. This record does not assert independent reproduction. ### Qwen3.5-397B Accuracy: 19.2. Result ID: 6a21799efbc373592c1ee456. [Reporting paper]() Recorded evaluation configuration: {"Harness":"direct"}. This record does not assert independent reproduction. ### GPT-5.2 Accuracy: 18.8. Result ID: 6a21799efbc373592c1ee464. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Harness":"terminus-2"}. This record does not assert independent reproduction. ### Qwen3.5-122B Accuracy: 15. Result ID: 6a21799efbc373592c1ee44d. [Reporting paper]() Recorded evaluation configuration: {"Harness":"direct"}. This record does not assert independent reproduction. ### Qwen3.5-35B Accuracy: 13.7. Result ID: 6a21799efbc373592c1ee452. [Reporting paper]() Recorded evaluation configuration: {"Harness":"direct"}. This record does not assert independent reproduction. ### Qwen3-235B Accuracy: 12.8. Result ID: 6a21799efbc373592c1ee45a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Harness":"direct"}. This record does not assert independent reproduction. ### Qwen3.5-122B Accuracy: 11.3. Result ID: 6a21799efbc373592c1ee451. [Reporting paper]() Recorded evaluation configuration: {"Harness":"claude-code"}. This record does not assert independent reproduction. ### Qwen3.5-397B Accuracy: 10. Result ID: 6a21799efbc373592c1ee459. [Reporting paper]() Recorded evaluation configuration: {"Harness":"claude-code"}. This record does not assert independent reproduction. ### Kimi-K2 Accuracy: 9. Result ID: 6a21799efbc373592c1ee45e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Harness":"direct"}. This record does not assert independent reproduction. ### Qwen3.5-35B Accuracy: 8.8. Result ID: 6a21799efbc373592c1ee455. [Reporting paper]() Recorded evaluation configuration: {"Harness":"claude-code"}. This record does not assert independent reproduction. ### Qwen3-Coder-480B Accuracy: 5. Result ID: 6a21799efbc373592c1ee44c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Harness":"claude-code"}. This record does not assert independent reproduction. ### Qwen3.5-122B Accuracy: 5. Result ID: 6a21799efbc373592c1ee450. [Reporting paper]() Recorded evaluation configuration: {"Harness":"terminus-kira"}. This record does not assert independent reproduction. ### Kimi-K2 Accuracy: 5. Result ID: 6a21799efbc373592c1ee461. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Harness":"terminus-kira"}. This record does not assert independent reproduction. ### Qwen3.5-122B Accuracy: 3.8. Result ID: 6a21799efbc373592c1ee44f. [Reporting paper]() Recorded evaluation configuration: {"Harness":"terminus-2"}. This record does not assert independent reproduction. ### Qwen3-235B Accuracy: 3.8. Result ID: 6a21799efbc373592c1ee45d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Harness":"terminus-kira"}. This record does not assert independent reproduction. ### GPT-5.2 Accuracy: 2.5. Result ID: 6a21799efbc373592c1ee462. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Harness":"direct"}. This record does not assert independent reproduction. ### Qwen3-Coder-480B Accuracy: 2.1. Result ID: 6a21799efbc373592c1ee448. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Harness":"direct"}. This record does not assert independent reproduction. ### Qwen3-Coder-480B Accuracy: 1.3. Result ID: 6a21799efbc373592c1ee44b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Harness":"terminus-kira"}. This record does not assert independent reproduction. ### Qwen3.5-122B Accuracy: 1.3. Result ID: 6a21799efbc373592c1ee44e. [Reporting paper]() Recorded evaluation configuration: {"Harness":"codex"}. This record does not assert independent reproduction. ### Qwen3.5-35B Accuracy: 1.3. Result ID: 6a21799efbc373592c1ee453. [Reporting paper]() Recorded evaluation configuration: {"Harness":"terminus-2"}. This record does not assert independent reproduction. ### Qwen3.5-35B Accuracy: 1.3. Result ID: 6a21799efbc373592c1ee454. [Reporting paper]() Recorded evaluation configuration: {"Harness":"terminus-kira"}. This record does not assert independent reproduction. ### Qwen3.5-397B Accuracy: 1.3. Result ID: 6a21799efbc373592c1ee457. [Reporting paper]() Recorded evaluation configuration: {"Harness":"terminus-2"}. This record does not assert independent reproduction. ### Qwen3-235B Accuracy: 1.3. Result ID: 6a21799efbc373592c1ee45c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Harness":"terminus-2"}. This record does not assert independent reproduction. ### Qwen3-Coder-480B Accuracy: 0. Result ID: 6a21799efbc373592c1ee449. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Harness":"codex"}. This record does not assert independent reproduction. ### Qwen3-Coder-480B Accuracy: 0. Result ID: 6a21799efbc373592c1ee44a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Harness":"terminus-2"}. This record does not assert independent reproduction. ### Qwen3.5-397B Accuracy: 0. Result ID: 6a21799efbc373592c1ee458. [Reporting paper]() Recorded evaluation configuration: {"Harness":"terminus-kira"}. This record does not assert independent reproduction. ### Qwen3-235B Accuracy: 0. Result ID: 6a21799efbc373592c1ee45b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Harness":"codex"}. This record does not assert independent reproduction. ### Kimi-K2 Accuracy: 0. Result ID: 6a21799efbc373592c1ee45f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Harness":"codex"}. This record does not assert independent reproduction. ### Kimi-K2 Accuracy: 0. Result ID: 6a21799efbc373592c1ee460. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Harness":"terminus-2"}. This record does not assert independent reproduction. ### GPT-5.2 Accuracy: 0. Result ID: 6a21799efbc373592c1ee463. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Harness":"codex"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.