# Implicit-conflict resolution on STALE 1.0 (full) Source: [Implicit-conflict resolution on STALE 1.0 (full)]() Collection index: [llms.txt]() Data updated: 2026-05-08T11:20:41.041Z Evaluation of model performance on implicit conflict resolution tasks using the full set of the STALE 1.0 dataset, measured by Type I/II SR, PR, and IPA Accuracies, and Overall Accuracy. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Implicit-conflict resolution | | Dataset (including any stated split) | STALE 1.0 (full) | | Metric | Type I SR Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 15 of 15 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Type I SR Accuracy](): Higher is better; unit: PERCENT. - [Type I PR Accuracy](): Higher is better; unit: PERCENT. - [Type I IPA Accuracy](): Higher is better; unit: PERCENT. - [Type II SR Accuracy](): Higher is better; unit: PERCENT. - [Type II PR Accuracy](): Higher is better; unit: PERCENT. - [Type II IPA Accuracy](): Higher is better; unit: PERCENT. - [Overall Accuracy](): Higher is better; unit: PERCENT. ## Recorded results | Method | Type I SR Accuracy | Result date | Report paper | | --- | --- | --- | --- | | Gemini-3.1-pro | 92 | Not recorded | 69fd6e5e5497b3b670c8975d | | CUPMEM | 91 | Not recorded | 69fd6e5e5497b3b670c8975d | | Qwen3.5-27B | 76 | Not recorded | 69fd6e5e5497b3b670c8975d | | LightMem | 52.5 | Not recorded | 69fd6e5e5497b3b670c8975d | | Gemini-3.1-flash-lite | 41 | Not recorded | 69fd6e5e5497b3b670c8975d | | Qwen3.5-9B | 36 | Not recorded | 69fd6e5e5497b3b670c8975d | | GPT-5.4 | 35 | Not recorded | 69fd6e5e5497b3b670c8975d | | GPT-4o-mini\* | 30 | Not recorded | 69fd6e5e5497b3b670c8975d | | GPT-5.4-nano | 20.5 | Not recorded | 69fd6e5e5497b3b670c8975d | | mem-0 | 17 | Not recorded | 69fd6e5e5497b3b670c8975d | | LiCoMemory | 15.5 | Not recorded | 69fd6e5e5497b3b670c8975d | | A-mem | 13.5 | Not recorded | 69fd6e5e5497b3b670c8975d | | MiniMax-M2.5 | 10.5 | Not recorded | 69fd6e5e5497b3b670c8975d | | Zep | 10 | Not recorded | 69fd6e5e5497b3b670c8975d | | Llama-3.3-70B-Instruct\* | 6.5 | Not recorded | 69fd6e5e5497b3b670c8975d | ### Gemini-3.1-pro Type I SR Accuracy: 92. Result ID: 69fdc6fc47f34ce924e1b0a0. [Reporting paper]() Recorded evaluation configuration: {"Category":"Closed-source LLMs"}. This record does not assert independent reproduction. ### CUPMEM Type I SR Accuracy: 91. Result ID: 69fdc6fc47f34ce924e1b0aa. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Category":"Memory Frameworks"}. This record does not assert independent reproduction. ### Qwen3.5-27B Type I SR Accuracy: 76. Result ID: 69fdc6fc47f34ce924e1b0a3. [Reporting paper]() Recorded evaluation configuration: {"Category":"Open-source LLMs"}. This record does not assert independent reproduction. ### LightMem Type I SR Accuracy: 52.5. Result ID: 69fdc6fc47f34ce924e1b0a5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Category":"Memory Frameworks"}. This record does not assert independent reproduction. ### Gemini-3.1-flash-lite Type I SR Accuracy: 41. Result ID: 69fdc6fc47f34ce924e1b09f. [Reporting paper]() Recorded evaluation configuration: {"Category":"Closed-source LLMs"}. This record does not assert independent reproduction. ### Qwen3.5-9B Type I SR Accuracy: 36. Result ID: 69fdc6fc47f34ce924e1b0a2. [Reporting paper]() Recorded evaluation configuration: {"Category":"Open-source LLMs"}. This record does not assert independent reproduction. ### GPT-5.4 Type I SR Accuracy: 35. Result ID: 69fdc6fc47f34ce924e1b09e. [Reporting paper]() Recorded evaluation configuration: {"Category":"Closed-source LLMs"}. This record does not assert independent reproduction. ### GPT-4o-mini\* Type I SR Accuracy: 30. Result ID: 69fdc6fc47f34ce924e1b09c. [Reporting paper]() Recorded evaluation configuration: {"Category":"Closed-source LLMs"}. This record does not assert independent reproduction. ### GPT-5.4-nano Type I SR Accuracy: 20.5. Result ID: 69fdc6fc47f34ce924e1b09d. [Reporting paper]() Recorded evaluation configuration: {"Category":"Closed-source LLMs"}. This record does not assert independent reproduction. ### mem-0 Type I SR Accuracy: 17. Result ID: 69fdc6fc47f34ce924e1b0a9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Category":"Memory Frameworks"}. This record does not assert independent reproduction. ### LiCoMemory Type I SR Accuracy: 15.5. Result ID: 69fdc6fc47f34ce924e1b0a7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Category":"Memory Frameworks"}. This record does not assert independent reproduction. ### A-mem Type I SR Accuracy: 13.5. Result ID: 69fdc6fc47f34ce924e1b0a8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Category":"Memory Frameworks"}. This record does not assert independent reproduction. ### MiniMax-M2.5 Type I SR Accuracy: 10.5. Result ID: 69fdc6fc47f34ce924e1b0a4. [Reporting paper]() Recorded evaluation configuration: {"Category":"Open-source LLMs"}. This record does not assert independent reproduction. ### Zep Type I SR Accuracy: 10. Result ID: 69fdc6fc47f34ce924e1b0a6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Category":"Memory Frameworks"}. This record does not assert independent reproduction. ### Llama-3.3-70B-Instruct\* Type I SR Accuracy: 6.5. Result ID: 69fdc6fc47f34ce924e1b0a1. [Reporting paper]() Recorded evaluation configuration: {"Category":"Open-source LLMs"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.