# Causal Reasoning on CLadder ID (held-out) Source: [Causal Reasoning on CLadder ID (held-out)]() Collection index: [llms.txt]() Data updated: 2026-06-16T08:44:06.551Z Evaluation of model performance on the held-out ID split of the CLadder dataset for the Causal Reasoning task, measured by P0, P1, and Gap metrics. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Causal Reasoning | | Dataset (including any stated split) | CLadder ID (held-out) | | Metric | P0 Score | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 13 of 13 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [P0 Score](): Higher is better; unit: PERCENT. - [P1 Score](): Higher is better; unit: PERCENT. - [Gap Score](): Lower is better; unit: SCALAR. ## Recorded results | Method | P0 Score | Result date | Report paper | | --- | --- | --- | --- | | Mistral-7B | 98 | Not recorded | 6a30d89fd33dfb23e783cc86 | | Qwen2.5-32B | 97 | Not recorded | 6a30d89fd33dfb23e783cc86 | | Qwen2.5-14B | 95.5 | Not recorded | 6a30d89fd33dfb23e783cc86 | | Llama-3.1-8B | 94.5 | Not recorded | 6a30d89fd33dfb23e783cc86 | | Qwen2.5-7B | 86.8 | Not recorded | 6a30d89fd33dfb23e783cc86 | | Qwen2.5-7B | 62.5 | Not recorded | 6a30d89fd33dfb23e783cc86 | | Qwen2.5-14B | 61.5 | Not recorded | 6a30d89fd33dfb23e783cc86 | | Mistral-7B | 61 | Not recorded | 6a30d89fd33dfb23e783cc86 | | Mistral-7B | 60.5 | Not recorded | 6a30d89fd33dfb23e783cc86 | | Phi-3-mini (3.8B) | 58.5 | Not recorded | 6a30d89fd33dfb23e783cc86 | | Qwen2.5-32B | 56.5 | Not recorded | 6a30d89fd33dfb23e783cc86 | | Phi-3-mini (3.8B) | 55 | Not recorded | 6a30d89fd33dfb23e783cc86 | | Llama-3.1-8B | 54 | Not recorded | 6a30d89fd33dfb23e783cc86 | ### Mistral-7B P0 Score: 98. Result ID: 6a310cc3fbc373592c2d67df. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Adapter type":"Vernier","recipe":"default","lr":"1e-4"}. This record does not assert independent reproduction. ### Qwen2.5-32B P0 Score: 97. Result ID: 6a310cc3fbc373592c2d67db. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Adapter type":"Vernier","QLoRA":"true"}. This record does not assert independent reproduction. ### Qwen2.5-14B P0 Score: 95.5. Result ID: 6a310cc3fbc373592c2d67d9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Adapter type":"Vernier","QLoRA":"true"}. This record does not assert independent reproduction. ### Llama-3.1-8B P0 Score: 94.5. Result ID: 6a310cc3fbc373592c2d67dd. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Adapter type":"Vernier"}. This record does not assert independent reproduction. ### Qwen2.5-7B P0 Score: 86.8. Result ID: 6a310cc3fbc373592c2d67d7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Adapter type":"Vernier"}. This record does not assert independent reproduction. ### Qwen2.5-7B P0 Score: 62.5. Result ID: 6a310cc3fbc373592c2d67d6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Adapter type":"Base"}. This record does not assert independent reproduction. ### Qwen2.5-14B P0 Score: 61.5. Result ID: 6a310cc3fbc373592c2d67d8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Adapter type":"Base","QLoRA":"true"}. This record does not assert independent reproduction. ### Mistral-7B P0 Score: 61. Result ID: 6a310cc3fbc373592c2d67de. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Adapter type":"Base"}. This record does not assert independent reproduction. ### Mistral-7B P0 Score: 60.5. Result ID: 6a310cc3fbc373592c2d67e0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Adapter type":"Vernier","recipe":"rescued","lr":"5e-6","early stopping step":"150"}. This record does not assert independent reproduction. ### Phi-3-mini (3.8B) P0 Score: 58.5. Result ID: 6a310cc3fbc373592c2d67d4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Adapter type":"Base"}. This record does not assert independent reproduction. ### Qwen2.5-32B P0 Score: 56.5. Result ID: 6a310cc3fbc373592c2d67da. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Adapter type":"Base","QLoRA":"true"}. This record does not assert independent reproduction. ### Phi-3-mini (3.8B) P0 Score: 55. Result ID: 6a310cc3fbc373592c2d67d5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Adapter type":"Vernier"}. This record does not assert independent reproduction. ### Llama-3.1-8B P0 Score: 54. Result ID: 6a310cc3fbc373592c2d67dc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Adapter type":"Base"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.