# Binary decision on PLASMA Source: [Binary decision on PLASMA]() Collection index: [llms.txt]() Data updated: 2026-04-15T09:22:41.333Z Evaluation of Binary decision performance on the PLASMA dataset, measured using accuracy. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Binary decision | | Dataset (including any stated split) | PLASMA | | Metric | Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 27 of 27 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Accuracy](): Higher is better; unit: PERCENT. ## Recorded results | Method | Accuracy | Result date | Report paper | | --- | --- | --- | --- | | IDEA | 84.5 | Not recorded | 69df1bb57d0172140ec7ba5e | | IDEA | 83.9 | Not recorded | 69df1bb57d0172140ec7ba5e | | CoT | 82.9 | Not recorded | 69df1bb57d0172140ec7ba5e | | Vanilla | 82 | Not recorded | 69df1bb57d0172140ec7ba5e | | IDEA w/o MC | 82 | Not recorded | 69df1bb57d0172140ec7ba5e | | Vanilla | 81.1 | Not recorded | 69df1bb57d0172140ec7ba5e | | IDEA w/o Inter | 80.2 | Not recorded | 69df1bb57d0172140ec7ba5e | | IDEA | 80.2 | Not recorded | 69df1bb57d0172140ec7ba5e | | BIRD | 79.9 | Not recorded | 69df1bb57d0172140ec7ba5e | | Logit | 79.7 | Not recorded | 69df1bb57d0172140ec7ba5e | | IDEA w/o MC | 79.5 | Not recorded | 69df1bb57d0172140ec7ba5e | | IDEA w/o Inter | 79.3 | Not recorded | 69df1bb57d0172140ec7ba5e | | Vanilla | 78.1 | Not recorded | 69df1bb57d0172140ec7ba5e | | IDEA w/o Inter | 76.8 | Not recorded | 69df1bb57d0172140ec7ba5e | | IDEA w/o EM | 76.6 | Not recorded | 69df1bb57d0172140ec7ba5e | | Logit | 76.4 | Not recorded | 69df1bb57d0172140ec7ba5e | | IDEA w/o EM | 76.3 | Not recorded | 69df1bb57d0172140ec7ba5e | | CoT | 75.9 | Not recorded | 69df1bb57d0172140ec7ba5e | | IDEA w/o EM | 75.7 | Not recorded | 69df1bb57d0172140ec7ba5e | | Logit | 75.6 | Not recorded | 69df1bb57d0172140ec7ba5e | | CoT | 75.4 | Not recorded | 69df1bb57d0172140ec7ba5e | | BIRD | 75.3 | Not recorded | 69df1bb57d0172140ec7ba5e | | IDEA w/o MC | 75.2 | Not recorded | 69df1bb57d0172140ec7ba5e | | CoT | 75.1 | Not recorded | 69df1bb57d0172140ec7ba5e | | CoT | 73.7 | Not recorded | 69df1bb57d0172140ec7ba5e | | BIRD | 73.3 | Not recorded | 69df1bb57d0172140ec7ba5e | | Random | 50 | Not recorded | 69df1bb57d0172140ec7ba5e | ### IDEA Accuracy: 84.5. Result ID: 69df580947f34ce924c85bc6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-32B"}. This record does not assert independent reproduction. ### IDEA Accuracy: 83.9. Result ID: 69df580947f34ce924c85bb6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-4B"}. This record does not assert independent reproduction. ### CoT Accuracy: 82.9. Result ID: 69df580947f34ce924c85bae. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-5.2"}. This record does not assert independent reproduction. ### Vanilla Accuracy: 82. Result ID: 69df580947f34ce924c85bb1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-4B"}. This record does not assert independent reproduction. ### IDEA w/o MC Accuracy: 82. Result ID: 69df580947f34ce924c85bc5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-32B"}. This record does not assert independent reproduction. ### Vanilla Accuracy: 81.1. Result ID: 69df580947f34ce924c85bc1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-32B"}. This record does not assert independent reproduction. ### IDEA w/o Inter Accuracy: 80.2. Result ID: 69df580947f34ce924c85bbb. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-8B"}. This record does not assert independent reproduction. ### IDEA Accuracy: 80.2. Result ID: 69df580947f34ce924c85bbe. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-8B"}. This record does not assert independent reproduction. ### BIRD Accuracy: 79.9. Result ID: 69df580947f34ce924c85bc2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-32B"}. This record does not assert independent reproduction. ### Logit Accuracy: 79.7. Result ID: 69df580947f34ce924c85bc0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-32B"}. This record does not assert independent reproduction. ### IDEA w/o MC Accuracy: 79.5. Result ID: 69df580947f34ce924c85bbd. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-8B"}. This record does not assert independent reproduction. ### IDEA w/o Inter Accuracy: 79.3. Result ID: 69df580947f34ce924c85bb3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-4B"}. This record does not assert independent reproduction. ### Vanilla Accuracy: 78.1. Result ID: 69df580947f34ce924c85bb9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-8B"}. This record does not assert independent reproduction. ### IDEA w/o Inter Accuracy: 76.8. Result ID: 69df580947f34ce924c85bc3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-32B"}. This record does not assert independent reproduction. ### IDEA w/o EM Accuracy: 76.6. Result ID: 69df580947f34ce924c85bbc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-8B"}. This record does not assert independent reproduction. ### Logit Accuracy: 76.4. Result ID: 69df580947f34ce924c85bb8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-8B"}. This record does not assert independent reproduction. ### IDEA w/o EM Accuracy: 76.3. Result ID: 69df580947f34ce924c85bc4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-32B"}. This record does not assert independent reproduction. ### CoT Accuracy: 75.9. Result ID: 69df580947f34ce924c85bb7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-8B"}. This record does not assert independent reproduction. ### IDEA w/o EM Accuracy: 75.7. Result ID: 69df580947f34ce924c85bb4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-4B"}. This record does not assert independent reproduction. ### Logit Accuracy: 75.6. Result ID: 69df580947f34ce924c85bb0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-4B"}. This record does not assert independent reproduction. ### CoT Accuracy: 75.4. Result ID: 69df580947f34ce924c85bad. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"DeepSeek R1"}. This record does not assert independent reproduction. ### BIRD Accuracy: 75.3. Result ID: 69df580947f34ce924c85bb2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-4B"}. This record does not assert independent reproduction. ### IDEA w/o MC Accuracy: 75.2. Result ID: 69df580947f34ce924c85bb5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-4B"}. This record does not assert independent reproduction. ### CoT Accuracy: 75.1. Result ID: 69df580947f34ce924c85baf. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-4B"}. This record does not assert independent reproduction. ### CoT Accuracy: 73.7. Result ID: 69df580947f34ce924c85bbf. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-32B"}. This record does not assert independent reproduction. ### BIRD Accuracy: 73.3. Result ID: 69df580947f34ce924c85bba. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-3-8B"}. This record does not assert independent reproduction. ### Random Accuracy: 50. Result ID: 69df580947f34ce924c85bac. [Reporting paper]() Recorded evaluation configuration: {"Model":"N/A"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.