Overdose Detection on JiraiBench 1.0 (test)
0.7743Macro F1S^3 Agent
Evaluation results
| Method | Links | |
|---|---|---|
| S^3 AgentLLM Backbone=DeepSeek V3.22026.01 | 0.7743 | |
| Self-RefineLLM Backbone=DeepSeek V3.22026.01 | 0.7652 | |
| SAS (English version)LLM Backbone=DeepSeek V3.22026.01 | 0.7505 | |
| Zero-shotLLM Backbone=DeepSeek V3.22026.01 | 0.7325 | |
| Zero-shot CoTLLM Backbone=DeepSeek V3.22026.01 | 0.7235 | |
| Plan-and-SolveLLM Backbone=DeepSeek V3.22026.01 | 0.7025 | |
| Self-RefineLLM Backbone=Gemma-3-12B-it2026.01 | 0.6549 | |
| OWLLLM Backbone=DeepSeek V3.22026.01 | 0.6397 | |
| Zero-shot CoTLLM Backbone=Ministral-3-8B2026.01 | 0.6338 | |
| Zero-shotLLM Backbone=Ministral-3-8B2026.01 | 0.6314 | |
| Plan-and-SolveLLM Backbone=Ministral-3-8B2026.01 | 0.6294 | |
| OWLLLM Backbone=Ministral-3-8B2026.01 | 0.5855 | |
| Self-RefineLLM Backbone=Qwen-2.5-7B2026.01 | 0.5785 | |
| S^3 AgentLLM Backbone=Gemma-3-12B-it2026.01 | 0.5769 | |
| S^3 AgentLLM Backbone=Ministral-3-8B2026.01 | 0.5727 | |
| SAS (English version)LLM Backbone=Gemma-3-12B-it2026.01 | 0.5685 | |
| SAS (English version)LLM Backbone=Ministral-3-8B2026.01 | 0.5672 | |
| SAS (English version)LLM Backbone=Qwen-2.5-7B2026.01 | 0.5613 | |
| OWLLLM Backbone=Gemma-3-12B-it2026.01 | 0.553 | |
| Self-RefineLLM Backbone=Ministral-3-8B2026.01 | 0.5483 | |
| Zero-shotLLM Backbone=Qwen-2.5-7B2026.01 | 0.5206 | |
| Zero-shotLLM Backbone=Gemma-3-12B-it2026.01 | 0.515 | |
| Zero-shot CoTLLM Backbone=Gemma-3-12B-it2026.01 | 0.509 | |
| S^3 AgentLLM Backbone=Qwen-2.5-7B2026.01 | 0.5084 | |
| Plan-and-SolveLLM Backbone=Gemma-3-12B-it2026.01 | 0.5082 | |
| OWLLLM Backbone=Qwen-2.5-7B2026.01 | 0.5015 | |
| Zero-shot CoTLLM Backbone=Qwen-2.5-7B2026.01 | 0.4759 | |
| Plan-and-SolveLLM Backbone=Qwen-2.5-7B2026.01 | 0.473 | |
| SAS (English version)LLM Backbone=Llama-3.1-8B2026.01 | 0.4384 | |
| Self-RefineLLM Backbone=Llama-3.1-8B2026.01 | 0.4221 | |
| Zero-shotLLM Backbone=Llama-3.1-8B2026.01 | 0.3622 | |
| S^3 AgentLLM Backbone=Llama-3.1-8B2026.01 | 0.3463 | |
| OWLLLM Backbone=Llama-3.1-8B2026.01 | 0.3457 | |
| Zero-shot CoTLLM Backbone=Llama-3.1-8B2026.01 | 0.3308 | |
| Plan-and-SolveLLM Backbone=Llama-3.1-8B2026.01 | 0.2842 |