# Dialogue-level classification on CRADLE-DIALOGUE Confirm-only class (test) Source: [Dialogue-level classification on CRADLE-DIALOGUE Confirm-only class (test)]() Collection index: [llms.txt]() Data updated: 2026-06-11T07:35:26.249Z Evaluation of dialogue-level classification performance on the test split of the CRADLE-DIALOGUE Confirm-only class dataset, measured using Jaccard Index, micro/macro F1, and micro/macro Recall. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Dialogue-level classification | | Dataset (including any stated split) | CRADLE-DIALOGUE Confirm-only class (test) | | Metric | Jaccard Index | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 15 of 15 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Jaccard Index](): Higher is better; unit: PERCENT. - [Micro F1](): Higher is better; unit: PERCENT. - [Macro F1](): Higher is better; unit: PERCENT. - [Micro Recall](): Higher is better; unit: PERCENT. - [Macro Recall](): Higher is better; unit: PERCENT. ## Recorded results | Method | Jaccard Index | Result date | Report paper | | --- | --- | --- | --- | | Claude-4.5-Sonnet | 79.4 | Not recorded | 6a28f31f90966ade919c1622 | | Qwen3-32b-FT | 79.28 | Not recorded | 6a28f31f90966ade919c1622 | | Gemini-3-Flash | 77.63 | Not recorded | 6a28f31f90966ade919c1622 | | GPT-5.1 | 76.75 | Not recorded | 6a28f31f90966ade919c1622 | | Qwen2.5-72b-FT | 75.83 | Not recorded | 6a28f31f90966ade919c1622 | | Llama3.3-70b-FT | 74.66 | Not recorded | 6a28f31f90966ade919c1622 | | gpt-oss-120b | 74.56 | Not recorded | 6a28f31f90966ade919c1622 | | Qwen3-32B | 73.49 | Not recorded | 6a28f31f90966ade919c1622 | | Qwen2.5-72B-Inst | 72.11 | Not recorded | 6a28f31f90966ade919c1622 | | Qwen3-14B | 70.65 | Not recorded | 6a28f31f90966ade919c1622 | | Llama-3.3-70B-Inst | 70.27 | Not recorded | 6a28f31f90966ade919c1622 | | Llama-4-Scout-17B | 68.9 | Not recorded | 6a28f31f90966ade919c1622 | | gpt-oss-20b | 67.83 | Not recorded | 6a28f31f90966ade919c1622 | | gemma-3-27b-it | 66 | Not recorded | 6a28f31f90966ade919c1622 | | gemma-3-12b-it | 64.37 | Not recorded | 6a28f31f90966ade919c1622 | ### Claude-4.5-Sonnet Jaccard Index: 79.4. Result ID: 6a2a6532fbc373592c27276c. [Reporting paper]() Recorded evaluation configuration: {"Model type":"Proprietary"}. This record does not assert independent reproduction. ### Qwen3-32b-FT Jaccard Index: 79.28. Result ID: 6a2a6532fbc373592c272771. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Fine-tuning source":"Dialogue"}. This record does not assert independent reproduction. ### Gemini-3-Flash Jaccard Index: 77.63. Result ID: 6a2a6532fbc373592c27276e. [Reporting paper]() Recorded evaluation configuration: {"Model type":"Proprietary"}. This record does not assert independent reproduction. ### GPT-5.1 Jaccard Index: 76.75. Result ID: 6a2a6532fbc373592c27276d. [Reporting paper]() Recorded evaluation configuration: {"Model type":"Proprietary"}. This record does not assert independent reproduction. ### Qwen2.5-72b-FT Jaccard Index: 75.83. Result ID: 6a2a6532fbc373592c272770. [Reporting paper]() Recorded evaluation configuration: {"Fine-tuning source":"Reddit Post"}. This record does not assert independent reproduction. ### Llama3.3-70b-FT Jaccard Index: 74.66. Result ID: 6a2a6532fbc373592c27276f. [Reporting paper]() Recorded evaluation configuration: {"Fine-tuning source":"Reddit Post"}. This record does not assert independent reproduction. ### gpt-oss-120b Jaccard Index: 74.56. Result ID: 6a2a6532fbc373592c27276b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model type":"Instruction-tuned"}. This record does not assert independent reproduction. ### Qwen3-32B Jaccard Index: 73.49. Result ID: 6a2a6532fbc373592c272768. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model type":"Instruction-tuned"}. This record does not assert independent reproduction. ### Qwen2.5-72B-Inst Jaccard Index: 72.11. Result ID: 6a2a6532fbc373592c272769. [Reporting paper]() Recorded evaluation configuration: {"Model type":"Instruction-tuned"}. This record does not assert independent reproduction. ### Qwen3-14B Jaccard Index: 70.65. Result ID: 6a2a6532fbc373592c272767. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model type":"Instruction-tuned"}. This record does not assert independent reproduction. ### Llama-3.3-70B-Inst Jaccard Index: 70.27. Result ID: 6a2a6532fbc373592c272763. [Reporting paper]() Recorded evaluation configuration: {"Model type":"Instruction-tuned"}. This record does not assert independent reproduction. ### Llama-4-Scout-17B Jaccard Index: 68.9. Result ID: 6a2a6532fbc373592c272764. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model type":"Instruction-tuned"}. This record does not assert independent reproduction. ### gpt-oss-20b Jaccard Index: 67.83. Result ID: 6a2a6532fbc373592c27276a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model type":"Instruction-tuned"}. This record does not assert independent reproduction. ### gemma-3-27b-it Jaccard Index: 66. Result ID: 6a2a6532fbc373592c272766. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model type":"Instruction-tuned"}. This record does not assert independent reproduction. ### gemma-3-12b-it Jaccard Index: 64.37. Result ID: 6a2a6532fbc373592c272765. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model type":"Instruction-tuned"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.