# Three-way probability ranking on COMMON2SENSE paired Source: [Three-way probability ranking on COMMON2SENSE paired]() Collection index: [llms.txt]() Data updated: 2026-04-15T09:22:41.398Z Evaluation of model performance on the COMMON2SENSE paired dataset using Three-way probability ranking, measured by F1 scores for categories C1, C2, Equality, and Macro F1. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Three-way probability ranking | | Dataset (including any stated split) | COMMON2SENSE paired | | Metric | F1 (C1) | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 30 of 30 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [F1 (C1)](): Higher is better; unit: PERCENT. - [F1 (C2)](): Higher is better; unit: PERCENT. - [F1 (Equality)](): Higher is better; unit: PERCENT. - [Macro F1](): Higher is better; unit: PERCENT. ## Recorded results | Method | F1 (C1) | Result date | Report paper | | --- | --- | --- | --- | | IDEA | 76.2 | Not recorded | 69df1bb57d0172140ec7ba5e | | IDEA w/o MC | 73.2 | Not recorded | 69df1bb57d0172140ec7ba5e | | IDEA | 72.8 | Not recorded | 69df1bb57d0172140ec7ba5e | | IDEA w/o Inter | 71.5 | Not recorded | 69df1bb57d0172140ec7ba5e | | IDEA w/o EM | 70.6 | Not recorded | 69df1bb57d0172140ec7ba5e | | IDEA w/o MC | 68.3 | Not recorded | 69df1bb57d0172140ec7ba5e | | IDEA w/o Inter | 67.5 | Not recorded | 69df1bb57d0172140ec7ba5e | | IDEA w/o EM | 65.8 | Not recorded | 69df1bb57d0172140ec7ba5e | | PWC | 61.5 | Not recorded | 69df1bb57d0172140ec7ba5e | | BIRD | 59 | Not recorded | 69df1bb57d0172140ec7ba5e | | IDEA | 57.5 | Not recorded | 69df1bb57d0172140ec7ba5e | | BIRD | 55.2 | Not recorded | 69df1bb57d0172140ec7ba5e | | PWC | 54.8 | Not recorded | 69df1bb57d0172140ec7ba5e | | PWC | 54.2 | Not recorded | 69df1bb57d0172140ec7ba5e | | IDEA w/o MC | 53.6 | Not recorded | 69df1bb57d0172140ec7ba5e | | IDEA w/o Inter | 52.2 | Not recorded | 69df1bb57d0172140ec7ba5e | | IDEA w/o EM | 50.5 | Not recorded | 69df1bb57d0172140ec7ba5e | | CoT | 45.5 | Not recorded | 69df1bb57d0172140ec7ba5e | | Vanilla | 43.8 | Not recorded | 69df1bb57d0172140ec7ba5e | | Vanilla | 41.8 | Not recorded | 69df1bb57d0172140ec7ba5e | | BIRD | 39.8 | Not recorded | 69df1bb57d0172140ec7ba5e | | Vanilla | 39.2 | Not recorded | 69df1bb57d0172140ec7ba5e | | CoT | 38.2 | Not recorded | 69df1bb57d0172140ec7ba5e | | CoT | 36.5 | Not recorded | 69df1bb57d0172140ec7ba5e | | CoT | 35.2 | Not recorded | 69df1bb57d0172140ec7ba5e | | Logit | 34.5 | Not recorded | 69df1bb57d0172140ec7ba5e | | Random | 33.3 | Not recorded | 69df1bb57d0172140ec7ba5e | | Logit | 32.2 | Not recorded | 69df1bb57d0172140ec7ba5e | | CoT | 31.8 | Not recorded | 69df1bb57d0172140ec7ba5e | | Logit | 28.1 | Not recorded | 69df1bb57d0172140ec7ba5e | ### IDEA F1 (C1): 76.2. Result ID: 69df580947f34ce924c85bff. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-32B"}. This record does not assert independent reproduction. ### IDEA w/o MC F1 (C1): 73.2. Result ID: 69df580947f34ce924c85bfe. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-32B"}. This record does not assert independent reproduction. ### IDEA F1 (C1): 72.8. Result ID: 69df580947f34ce924c85bf6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-8B"}. This record does not assert independent reproduction. ### IDEA w/o Inter F1 (C1): 71.5. Result ID: 69df580947f34ce924c85bfc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-32B"}. This record does not assert independent reproduction. ### IDEA w/o EM F1 (C1): 70.6. Result ID: 69df580947f34ce924c85bfd. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-32B"}. This record does not assert independent reproduction. ### IDEA w/o MC F1 (C1): 68.3. Result ID: 69df580947f34ce924c85bf5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-8B"}. This record does not assert independent reproduction. ### IDEA w/o Inter F1 (C1): 67.5. Result ID: 69df580947f34ce924c85bf3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-8B"}. This record does not assert independent reproduction. ### IDEA w/o EM F1 (C1): 65.8. Result ID: 69df580947f34ce924c85bf4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-8B"}. This record does not assert independent reproduction. ### PWC F1 (C1): 61.5. Result ID: 69df580947f34ce924c85bf2. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen-3-8B"}. This record does not assert independent reproduction. ### BIRD F1 (C1): 59. Result ID: 69df580947f34ce924c85bfa. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-32B"}. This record does not assert independent reproduction. ### IDEA F1 (C1): 57.5. Result ID: 69df580947f34ce924c85bed. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-4B"}. This record does not assert independent reproduction. ### BIRD F1 (C1): 55.2. Result ID: 69df580947f34ce924c85bf1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-8B"}. This record does not assert independent reproduction. ### PWC F1 (C1): 54.8. Result ID: 69df580947f34ce924c85bfb. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen-3-32B"}. This record does not assert independent reproduction. ### PWC F1 (C1): 54.2. Result ID: 69df580947f34ce924c85be9. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen-3-4B"}. This record does not assert independent reproduction. ### IDEA w/o MC F1 (C1): 53.6. Result ID: 69df580947f34ce924c85bec. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-4B"}. This record does not assert independent reproduction. ### IDEA w/o Inter F1 (C1): 52.2. Result ID: 69df580947f34ce924c85bea. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-4B"}. This record does not assert independent reproduction. ### IDEA w/o EM F1 (C1): 50.5. Result ID: 69df580947f34ce924c85beb. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-4B"}. This record does not assert independent reproduction. ### CoT F1 (C1): 45.5. Result ID: 69df580947f34ce924c85be4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-5.2"}. This record does not assert independent reproduction. ### Vanilla F1 (C1): 43.8. Result ID: 69df580947f34ce924c85bf9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-32B"}. This record does not assert independent reproduction. ### Vanilla F1 (C1): 41.8. Result ID: 69df580947f34ce924c85bf0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-8B"}. This record does not assert independent reproduction. ### BIRD F1 (C1): 39.8. Result ID: 69df580947f34ce924c85be8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-4B"}. This record does not assert independent reproduction. ### Vanilla F1 (C1): 39.2. Result ID: 69df580947f34ce924c85be7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-4B"}. This record does not assert independent reproduction. ### CoT F1 (C1): 38.2. Result ID: 69df580947f34ce924c85bf7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-32B"}. This record does not assert independent reproduction. ### CoT F1 (C1): 36.5. Result ID: 69df580947f34ce924c85bee. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-8B"}. This record does not assert independent reproduction. ### CoT F1 (C1): 35.2. Result ID: 69df580947f34ce924c85be5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-4B"}. This record does not assert independent reproduction. ### Logit F1 (C1): 34.5. Result ID: 69df580947f34ce924c85bf8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-32B"}. This record does not assert independent reproduction. ### Random F1 (C1): 33.3. Result ID: 69df580947f34ce924c85be2. [Reporting paper]() Recorded evaluation configuration: {"Model":"N/A"}. This record does not assert independent reproduction. ### Logit F1 (C1): 32.2. Result ID: 69df580947f34ce924c85bef. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-8B"}. This record does not assert independent reproduction. ### CoT F1 (C1): 31.8. Result ID: 69df580947f34ce924c85be3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"DeepSeek R1"}. This record does not assert independent reproduction. ### Logit F1 (C1): 28.1. Result ID: 69df580947f34ce924c85be6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-4B"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.