# 3-category taxonomy classification (multiple responses allowed) on Mutation Dataset (Experiment C) Source: [3-category taxonomy classification (multiple responses allowed) on Mutation Dataset (Experiment C)]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:34:46.766Z Evaluation of 3-category taxonomy classification performance on the Mutation Dataset (Experiment C), allowing for multiple responses per sample. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | 3-category taxonomy classification (multiple responses allowed) | | Dataset (including any stated split) | Mutation Dataset (Experiment C) | | Metric | Agreement Coefficient (AC) | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 12 of 12 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Agreement Coefficient (AC)](): Higher is better; unit: PERCENT. - [True Classification Rate (TC)](): Higher is better; unit: PERCENT. - [Category Coverage (CC)](): Higher is better; unit: PERCENT. - [Total Correct Rate](): Higher is better; unit: PERCENT. ## Recorded results | Method | Agreement Coefficient (AC) | Result date | Report paper | | --- | --- | --- | --- | | Deepseek-r1-7b | 97.64 | Not recorded | 695b456353fdd5c361742f15 | | Deepseek-r1-7b | 94.68 | Not recorded | 695b456353fdd5c361742f15 | | Llama-8b | 84.93 | Not recorded | 695b456353fdd5c361742f15 | | Llama-70b | 79.76 | Not recorded | 695b456353fdd5c361742f15 | | Llama-8b | 77.55 | Not recorded | 695b456353fdd5c361742f15 | | Llama-70b | 75.63 | Not recorded | 695b456353fdd5c361742f15 | | Deepseek-r1-7b | 66.47 | Not recorded | 695b456353fdd5c361742f15 | | Llama-70b | 65.58 | Not recorded | 695b456353fdd5c361742f15 | | Gemini-2.5-Pro | 59.53 | Not recorded | 695b456353fdd5c361742f15 | | Gemini-2.5-Pro | 57.02 | Not recorded | 695b456353fdd5c361742f15 | | Llama-8b | 55.39 | Not recorded | 695b456353fdd5c361742f15 | | Gemini-2.5-Pro | 49.93 | Not recorded | 695b456353fdd5c361742f15 | ### Deepseek-r1-7b Agreement Coefficient (AC): 97.64. Result ID: 6996f7605698665ec4441b23. [Reporting paper]() Recorded evaluation configuration: {"Few-shot setting":"One-Shot"}. This record does not assert independent reproduction. ### Deepseek-r1-7b Agreement Coefficient (AC): 94.68. Result ID: 6996f7605698665ec4441b22. [Reporting paper]() Recorded evaluation configuration: {"Few-shot setting":"Zero-Shot"}. This record does not assert independent reproduction. ### Llama-8b Agreement Coefficient (AC): 84.93. Result ID: 6996f7605698665ec4441b19. [Reporting paper]() Recorded evaluation configuration: {"Few-shot setting":"Zero-Shot"}. This record does not assert independent reproduction. ### Llama-70b Agreement Coefficient (AC): 79.76. Result ID: 6996f7605698665ec4441b1c. [Reporting paper]() Recorded evaluation configuration: {"Few-shot setting":"Zero-Shot"}. This record does not assert independent reproduction. ### Llama-8b Agreement Coefficient (AC): 77.55. Result ID: 6996f7605698665ec4441b1a. [Reporting paper]() Recorded evaluation configuration: {"Few-shot setting":"One-Shot"}. This record does not assert independent reproduction. ### Llama-70b Agreement Coefficient (AC): 75.63. Result ID: 6996f7605698665ec4441b1d. [Reporting paper]() Recorded evaluation configuration: {"Few-shot setting":"One-Shot"}. This record does not assert independent reproduction. ### Deepseek-r1-7b Agreement Coefficient (AC): 66.47. Result ID: 6996f7605698665ec4441b24. [Reporting paper]() Recorded evaluation configuration: {"Few-shot setting":"Two-Shot"}. This record does not assert independent reproduction. ### Llama-70b Agreement Coefficient (AC): 65.58. Result ID: 6996f7605698665ec4441b1e. [Reporting paper]() Recorded evaluation configuration: {"Few-shot setting":"Two-Shot"}. This record does not assert independent reproduction. ### Gemini-2.5-Pro Agreement Coefficient (AC): 59.53. Result ID: 6996f7605698665ec4441b1f. [Reporting paper]() Recorded evaluation configuration: {"Few-shot setting":"Zero-Shot"}. This record does not assert independent reproduction. ### Gemini-2.5-Pro Agreement Coefficient (AC): 57.02. Result ID: 6996f7605698665ec4441b20. [Reporting paper]() Recorded evaluation configuration: {"Few-shot setting":"One-Shot"}. This record does not assert independent reproduction. ### Llama-8b Agreement Coefficient (AC): 55.39. Result ID: 6996f7605698665ec4441b1b. [Reporting paper]() Recorded evaluation configuration: {"Few-shot setting":"Two-Shot"}. This record does not assert independent reproduction. ### Gemini-2.5-Pro Agreement Coefficient (AC): 49.93. Result ID: 6996f7605698665ec4441b21. [Reporting paper]() Recorded evaluation configuration: {"Few-shot setting":"Two-Shot"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.