# Transitive Reasoning on Mammal Source: [Transitive Reasoning on Mammal]() Collection index: [llms.txt]() Data updated: 2026-04-15T07:12:08.948Z Evaluation of model performance on the Transitive Reasoning task using the Mammal dataset, assessed using metrics like F1 score. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Transitive reasoning | | Dataset (including any stated split) | Mammal | | Metric | F1 Score | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 18 of 18 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [F1 Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | F1 Score | Result date | Report paper | | --- | --- | --- | --- | | RegD | 71.8 | Not recorded | 68fa7bf31e209131cea8b85b | | RegD | 71.6 | Not recorded | 68fa7bf31e209131cea8b85b | | ShadowCone\* | 69.4 | Not recorded | 68fa7bf31e209131cea8b85b | | ShadowCone\* | 67.8 | Not recorded | 68fa7bf31e209131cea8b85b | | RegD | 64.9 | Not recorded | 68fa7bf31e209131cea8b85b | | RegD | 62.7 | Not recorded | 68fa7bf31e209131cea8b85b | | ShadowCone\* | 57.7 | Not recorded | 68fa7bf31e209131cea8b85b | | EntailmentCone\* | 56.3 | Not recorded | 68fa7bf31e209131cea8b85b | | EntailmentCone\* | 54.4 | Not recorded | 68fa7bf31e209131cea8b85b | | ShadowCone\* | 52.8 | Not recorded | 68fa7bf31e209131cea8b85b | | ELBE | 36.8 | Not recorded | 68fa7bf31e209131cea8b85b | | τBox | 33.5 | Not recorded | 68fa7bf31e209131cea8b85b | | OE | 31 | Not recorded | 68fa7bf31e209131cea8b85b | | ELBE | 30.3 | Not recorded | 68fa7bf31e209131cea8b85b | | τBox | 29 | Not recorded | 68fa7bf31e209131cea8b85b | | ELEM | 28.8 | Not recorded | 68fa7bf31e209131cea8b85b | | ELEM | 27.7 | Not recorded | 68fa7bf31e209131cea8b85b | | OE | 25.4 | Not recorded | 68fa7bf31e209131cea8b85b | ### RegD F1 Score: 71.8. Result ID: 69df392e47f34ce924c786b6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Variant":"ball","d":"5"}. This record does not assert independent reproduction. ### RegD F1 Score: 71.6. Result ID: 69df392e47f34ce924c786b4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Variant":"box","d":"5"}. This record does not assert independent reproduction. ### ShadowCone\* F1 Score: 69.4. Result ID: 69df392e47f34ce924c786b0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Variant":"Umbral-half","d":"5"}. This record does not assert independent reproduction. ### ShadowCone\* F1 Score: 67.8. Result ID: 69df392e47f34ce924c786b2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Variant":"Penumbral-half","d":"5"}. This record does not assert independent reproduction. ### RegD F1 Score: 64.9. Result ID: 69df392e47f34ce924c786b3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Variant":"box","d":"2"}. This record does not assert independent reproduction. ### RegD F1 Score: 62.7. Result ID: 69df392e47f34ce924c786b5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Variant":"ball","d":"2"}. This record does not assert independent reproduction. ### ShadowCone\* F1 Score: 57.7. Result ID: 69df392e47f34ce924c786af. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Variant":"Umbral-half","d":"2"}. This record does not assert independent reproduction. ### EntailmentCone\* F1 Score: 56.3. Result ID: 69df392e47f34ce924c786ae. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"d":"5"}. This record does not assert independent reproduction. ### EntailmentCone\* F1 Score: 54.4. Result ID: 69df392e47f34ce924c786ad. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"d":"2"}. This record does not assert independent reproduction. ### ShadowCone\* F1 Score: 52.8. Result ID: 69df392e47f34ce924c786b1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Variant":"Penumbral-half","d":"2"}. This record does not assert independent reproduction. ### ELBE F1 Score: 36.8. Result ID: 69df392e47f34ce924c786aa. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Variant":"box baseline","d":"5"}. This record does not assert independent reproduction. ### τBox F1 Score: 33.5. Result ID: 69df392e47f34ce924c786a6. [Reporting paper]() Recorded evaluation configuration: {"d":"5"}. This record does not assert independent reproduction. ### OE F1 Score: 31. Result ID: 69df392e47f34ce924c786a8. [Reporting paper]() Recorded evaluation configuration: {"d":"5"}. This record does not assert independent reproduction. ### ELBE F1 Score: 30.3. Result ID: 69df392e47f34ce924c786a9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Variant":"box baseline","d":"2"}. This record does not assert independent reproduction. ### τBox F1 Score: 29. Result ID: 69df392e47f34ce924c786a5. [Reporting paper]() Recorded evaluation configuration: {"d":"2"}. This record does not assert independent reproduction. ### ELEM F1 Score: 28.8. Result ID: 69df392e47f34ce924c786ac. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Variant":"ball baseline","d":"5"}. This record does not assert independent reproduction. ### ELEM F1 Score: 27.7. Result ID: 69df392e47f34ce924c786ab. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Variant":"ball baseline","d":"2"}. This record does not assert independent reproduction. ### OE F1 Score: 25.4. Result ID: 69df392e47f34ce924c786a7. [Reporting paper]() Recorded evaluation configuration: {"d":"2"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.