# Overdose Detection on JiraiBench 1.0 (test) Source: [Overdose Detection on JiraiBench 1.0 (test)]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:51:27.767Z Evaluation of model performance on the test split of the JiraiBench 1.0 dataset for the task of Overdose Detection, reporting Macro F1 score. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Overdose Detection | | Dataset (including any stated split) | JiraiBench 1.0 (test) | | Metric | Macro F1 | | Unit | SCALAR | | Direction | Higher is better | | Results loaded | 35 of 35 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Macro F1](): Higher is better; unit: SCALAR. ## Recorded results | Method | Macro F1 | Result date | Report paper | | --- | --- | --- | --- | | S^3 Agent | 0.7743 | Not recorded | 69608b7653fdd5c3617443a2 | | Self-Refine | 0.7652 | Not recorded | 69608b7653fdd5c3617443a2 | | SAS (English version) | 0.7505 | Not recorded | 69608b7653fdd5c3617443a2 | | Zero-shot | 0.7325 | Not recorded | 69608b7653fdd5c3617443a2 | | Zero-shot CoT | 0.7235 | Not recorded | 69608b7653fdd5c3617443a2 | | Plan-and-Solve | 0.7025 | Not recorded | 69608b7653fdd5c3617443a2 | | Self-Refine | 0.6549 | Not recorded | 69608b7653fdd5c3617443a2 | | OWL | 0.6397 | Not recorded | 69608b7653fdd5c3617443a2 | | Zero-shot CoT | 0.6338 | Not recorded | 69608b7653fdd5c3617443a2 | | Zero-shot | 0.6314 | Not recorded | 69608b7653fdd5c3617443a2 | | Plan-and-Solve | 0.6294 | Not recorded | 69608b7653fdd5c3617443a2 | | OWL | 0.5855 | Not recorded | 69608b7653fdd5c3617443a2 | | Self-Refine | 0.5785 | Not recorded | 69608b7653fdd5c3617443a2 | | S^3 Agent | 0.5769 | Not recorded | 69608b7653fdd5c3617443a2 | | S^3 Agent | 0.5727 | Not recorded | 69608b7653fdd5c3617443a2 | | SAS (English version) | 0.5685 | Not recorded | 69608b7653fdd5c3617443a2 | | SAS (English version) | 0.5672 | Not recorded | 69608b7653fdd5c3617443a2 | | SAS (English version) | 0.5613 | Not recorded | 69608b7653fdd5c3617443a2 | | OWL | 0.553 | Not recorded | 69608b7653fdd5c3617443a2 | | Self-Refine | 0.5483 | Not recorded | 69608b7653fdd5c3617443a2 | | Zero-shot | 0.5206 | Not recorded | 69608b7653fdd5c3617443a2 | | Zero-shot | 0.515 | Not recorded | 69608b7653fdd5c3617443a2 | | Zero-shot CoT | 0.509 | Not recorded | 69608b7653fdd5c3617443a2 | | S^3 Agent | 0.5084 | Not recorded | 69608b7653fdd5c3617443a2 | | Plan-and-Solve | 0.5082 | Not recorded | 69608b7653fdd5c3617443a2 | | OWL | 0.5015 | Not recorded | 69608b7653fdd5c3617443a2 | | Zero-shot CoT | 0.4759 | Not recorded | 69608b7653fdd5c3617443a2 | | Plan-and-Solve | 0.473 | Not recorded | 69608b7653fdd5c3617443a2 | | SAS (English version) | 0.4384 | Not recorded | 69608b7653fdd5c3617443a2 | | Self-Refine | 0.4221 | Not recorded | 69608b7653fdd5c3617443a2 | | Zero-shot | 0.3622 | Not recorded | 69608b7653fdd5c3617443a2 | | S^3 Agent | 0.3463 | Not recorded | 69608b7653fdd5c3617443a2 | | OWL | 0.3457 | Not recorded | 69608b7653fdd5c3617443a2 | | Zero-shot CoT | 0.3308 | Not recorded | 69608b7653fdd5c3617443a2 | | Plan-and-Solve | 0.2842 | Not recorded | 69608b7653fdd5c3617443a2 | ### S^3 Agent Macro F1: 0.7743. Result ID: 699696705698665ec4dfe572. [Reporting paper]() Recorded evaluation configuration: {"LLM Backbone":"DeepSeek V3.2"}. This record does not assert independent reproduction. ### Self-Refine Macro F1: 0.7652. Result ID: 699696705698665ec4dfe56d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM Backbone":"DeepSeek V3.2"}. This record does not assert independent reproduction. ### SAS (English version) Macro F1: 0.7505. Result ID: 699696705698665ec4dfe57c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM Backbone":"DeepSeek V3.2"}. This record does not assert independent reproduction. ### Zero-shot Macro F1: 0.7325. Result ID: 699696705698665ec4dfe55e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM Backbone":"DeepSeek V3.2"}. This record does not assert independent reproduction. ### Zero-shot CoT Macro F1: 0.7235. Result ID: 699696705698665ec4dfe563. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"LLM Backbone":"DeepSeek V3.2"}. This record does not assert independent reproduction. ### Plan-and-Solve Macro F1: 0.7025. Result ID: 699696705698665ec4dfe568. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM Backbone":"DeepSeek V3.2"}. This record does not assert independent reproduction. ### Self-Refine Macro F1: 0.6549. Result ID: 699696705698665ec4dfe56f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM Backbone":"Gemma-3-12B-it"}. This record does not assert independent reproduction. ### OWL Macro F1: 0.6397. Result ID: 699696705698665ec4dfe577. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM Backbone":"DeepSeek V3.2"}. This record does not assert independent reproduction. ### Zero-shot CoT Macro F1: 0.6338. Result ID: 699696705698665ec4dfe564. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"LLM Backbone":"Ministral-3-8B"}. This record does not assert independent reproduction. ### Zero-shot Macro F1: 0.6314. Result ID: 699696705698665ec4dfe55f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM Backbone":"Ministral-3-8B"}. This record does not assert independent reproduction. ### Plan-and-Solve Macro F1: 0.6294. Result ID: 699696705698665ec4dfe569. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM Backbone":"Ministral-3-8B"}. This record does not assert independent reproduction. ### OWL Macro F1: 0.5855. Result ID: 699696705698665ec4dfe578. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM Backbone":"Ministral-3-8B"}. This record does not assert independent reproduction. ### Self-Refine Macro F1: 0.5785. Result ID: 699696705698665ec4dfe56b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM Backbone":"Qwen-2.5-7B"}. This record does not assert independent reproduction. ### S^3 Agent Macro F1: 0.5769. Result ID: 699696705698665ec4dfe574. [Reporting paper]() Recorded evaluation configuration: {"LLM Backbone":"Gemma-3-12B-it"}. This record does not assert independent reproduction. ### S^3 Agent Macro F1: 0.5727. Result ID: 699696705698665ec4dfe573. [Reporting paper]() Recorded evaluation configuration: {"LLM Backbone":"Ministral-3-8B"}. This record does not assert independent reproduction. ### SAS (English version) Macro F1: 0.5685. Result ID: 699696705698665ec4dfe57e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM Backbone":"Gemma-3-12B-it"}. This record does not assert independent reproduction. ### SAS (English version) Macro F1: 0.5672. Result ID: 699696705698665ec4dfe57d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM Backbone":"Ministral-3-8B"}. This record does not assert independent reproduction. ### SAS (English version) Macro F1: 0.5613. Result ID: 699696705698665ec4dfe57a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM Backbone":"Qwen-2.5-7B"}. This record does not assert independent reproduction. ### OWL Macro F1: 0.553. Result ID: 699696705698665ec4dfe579. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM Backbone":"Gemma-3-12B-it"}. This record does not assert independent reproduction. ### Self-Refine Macro F1: 0.5483. Result ID: 699696705698665ec4dfe56e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM Backbone":"Ministral-3-8B"}. This record does not assert independent reproduction. ### Zero-shot Macro F1: 0.5206. Result ID: 699696705698665ec4dfe55c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM Backbone":"Qwen-2.5-7B"}. This record does not assert independent reproduction. ### Zero-shot Macro F1: 0.515. Result ID: 699696705698665ec4dfe560. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM Backbone":"Gemma-3-12B-it"}. This record does not assert independent reproduction. ### Zero-shot CoT Macro F1: 0.509. Result ID: 699696705698665ec4dfe565. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"LLM Backbone":"Gemma-3-12B-it"}. This record does not assert independent reproduction. ### S^3 Agent Macro F1: 0.5084. Result ID: 699696705698665ec4dfe570. [Reporting paper]() Recorded evaluation configuration: {"LLM Backbone":"Qwen-2.5-7B"}. This record does not assert independent reproduction. ### Plan-and-Solve Macro F1: 0.5082. Result ID: 699696705698665ec4dfe56a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM Backbone":"Gemma-3-12B-it"}. This record does not assert independent reproduction. ### OWL Macro F1: 0.5015. Result ID: 699696705698665ec4dfe575. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM Backbone":"Qwen-2.5-7B"}. This record does not assert independent reproduction. ### Zero-shot CoT Macro F1: 0.4759. Result ID: 699696705698665ec4dfe561. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"LLM Backbone":"Qwen-2.5-7B"}. This record does not assert independent reproduction. ### Plan-and-Solve Macro F1: 0.473. Result ID: 699696705698665ec4dfe566. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM Backbone":"Qwen-2.5-7B"}. This record does not assert independent reproduction. ### SAS (English version) Macro F1: 0.4384. Result ID: 699696705698665ec4dfe57b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM Backbone":"Llama-3.1-8B"}. This record does not assert independent reproduction. ### Self-Refine Macro F1: 0.4221. Result ID: 699696705698665ec4dfe56c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM Backbone":"Llama-3.1-8B"}. This record does not assert independent reproduction. ### Zero-shot Macro F1: 0.3622. Result ID: 699696705698665ec4dfe55d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM Backbone":"Llama-3.1-8B"}. This record does not assert independent reproduction. ### S^3 Agent Macro F1: 0.3463. Result ID: 699696705698665ec4dfe571. [Reporting paper]() Recorded evaluation configuration: {"LLM Backbone":"Llama-3.1-8B"}. This record does not assert independent reproduction. ### OWL Macro F1: 0.3457. Result ID: 699696705698665ec4dfe576. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM Backbone":"Llama-3.1-8B"}. This record does not assert independent reproduction. ### Zero-shot CoT Macro F1: 0.3308. Result ID: 699696705698665ec4dfe562. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"LLM Backbone":"Llama-3.1-8B"}. This record does not assert independent reproduction. ### Plan-and-Solve Macro F1: 0.2842. Result ID: 699696705698665ec4dfe567. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM Backbone":"Llama-3.1-8B"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.