# Inter-Context Knowledge Conflict on PopQA Source: [Inter-Context Knowledge Conflict on PopQA]() Collection index: [llms.txt]() Data updated: 2026-07-01T07:46:48.218Z Evaluation of model performance on the PopQA dataset focusing on Inter-Context Knowledge Conflict resolution, measured using ACCL, ACCEM, and ACCCEM. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Inter-Context Knowledge Conflict | | Dataset (including any stated split) | popQA | | Metric | ACCL | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 32 of 32 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [ACCL](): Higher is better; unit: PERCENT. - [ACCEM](): Higher is better; unit: PERCENT. - [ACCCEM](): Higher is better; unit: PERCENT. ## Recorded results | Method | ACCL | Result date | Report paper | | --- | --- | --- | --- | | KCR (“Graph”) | 90.28 | Not recorded | 68fa7a621e209131cea82c1f | | KCR (“Graph”) | 87.5 | Not recorded | 68fa7a621e209131cea82c1f | | KCR (“Text”) | 86.67 | Not recorded | 68fa7a621e209131cea82c1f | | KCR (“Text”) | 84.17 | Not recorded | 68fa7a621e209131cea82c1f | | KCR (“Graph”) | 80.14 | Not recorded | 68fa7a621e209131cea82c1f | | KCR (“Text”) | 74.72 | Not recorded | 68fa7a621e209131cea82c1f | | KCR (“Graph”) | 69.58 | Not recorded | 68fa7a621e209131cea82c1f | | KCR (“Text”) | 68.61 | Not recorded | 68fa7a621e209131cea82c1f | | ASTUTE RAG (GPT-5.1) | 66.94 | Not recorded | 68fa7a621e209131cea82c1f | | Llama 3.1-8B | 56.53 | Not recorded | 68fa7a621e209131cea82c1f | | ASTUTE RAG (GPT-4o) | 54.86 | Not recorded | 68fa7a621e209131cea82c1f | | Directly Use Backbone | 54.31 | Not recorded | 68fa7a621e209131cea82c1f | | Directly Use Backbone | 54.31 | Not recorded | 68fa7a621e209131cea82c1f | | Qwen 2.5-3B | 51.67 | Not recorded | 68fa7a621e209131cea82c1f | | ASTUTE RAG | 50.28 | Not recorded | 68fa7a621e209131cea82c1f | | Directly Use Backbone | 49.58 | Not recorded | 68fa7a621e209131cea82c1f | | GPT-5.1 (Thinking) | 49.44 | Not recorded | 68fa7a621e209131cea82c1f | | Directly Use Backbone | 48.47 | Not recorded | 68fa7a621e209131cea82c1f | | ASTUTE RAG | 43.75 | Not recorded | 68fa7a621e209131cea82c1f | | CAD Decoding | 40.28 | Not recorded | 68fa7a621e209131cea82c1f | | CAD Decoding | 40.28 | Not recorded | 68fa7a621e209131cea82c1f | | Qwen 2.5-1.5B | 39.86 | Not recorded | 68fa7a621e209131cea82c1f | | ASTUTE RAG | 39.72 | Not recorded | 68fa7a621e209131cea82c1f | | CUAD (Default) | 34.31 | Not recorded | 68fa7a621e209131cea82c1f | | ADACAD (Default) | 34.31 | Not recorded | 68fa7a621e209131cea82c1f | | COCOA (Default) | 34.31 | Not recorded | 68fa7a621e209131cea82c1f | | GPT-4o-mini | 30.42 | Not recorded | 68fa7a621e209131cea82c1f | | Llama 3.2-1B | 30 | Not recorded | 68fa7a621e209131cea82c1f | | CAD Decoding | 29.72 | Not recorded | 68fa7a621e209131cea82c1f | | Llama 3.2-3B | 17.64 | Not recorded | 68fa7a621e209131cea82c1f | | Qwen 3-8B (Thinking) | 17.36 | Not recorded | 68fa7a621e209131cea82c1f | | Qwen 2.5-7B | 12.5 | Not recorded | 68fa7a621e209131cea82c1f | ### KCR (“Graph”) ACCL: 90.28. Result ID: 6a44c5e773d097d845beb6dc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen 3-8B","Format":"Graph"}. This record does not assert independent reproduction. ### KCR (“Graph”) ACCL: 87.5. Result ID: 6a44c5e773d097d845beb6cf. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama 3.2-3B","Format":"Graph"}. This record does not assert independent reproduction. ### KCR (“Text”) ACCL: 86.67. Result ID: 6a44c5e773d097d845beb6d3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen 2.5-7B","Format":"Text"}. This record does not assert independent reproduction. ### KCR (“Text”) ACCL: 84.17. Result ID: 6a44c5e773d097d845beb6ce. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama 3.2-3B","Format":"Text"}. This record does not assert independent reproduction. ### KCR (“Graph”) ACCL: 80.14. Result ID: 6a44c5e773d097d845beb6d4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen 2.5-7B","Format":"Graph"}. This record does not assert independent reproduction. ### KCR (“Text”) ACCL: 74.72. Result ID: 6a44c5e773d097d845beb6db. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen 3-8B","Format":"Text"}. This record does not assert independent reproduction. ### KCR (“Graph”) ACCL: 69.58. Result ID: 6a44c5e773d097d845beb6d7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen 3-8B (Thinking)","Format":"Graph"}. This record does not assert independent reproduction. ### KCR (“Text”) ACCL: 68.61. Result ID: 6a44c5e773d097d845beb6d6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen 3-8B (Thinking)","Format":"Text"}. This record does not assert independent reproduction. ### ASTUTE RAG (GPT-5.1) ACCL: 66.94. Result ID: 6a44c5e773d097d845beb6ca. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Family":"General Baselines (With Context)"}. This record does not assert independent reproduction. ### Llama 3.1-8B ACCL: 56.53. Result ID: 6a44c5e773d097d845beb6c3. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Family":"General Baselines (With Context)"}. This record does not assert independent reproduction. ### ASTUTE RAG (GPT-4o) ACCL: 54.86. Result ID: 6a44c5e773d097d845beb6c9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Family":"General Baselines (With Context)"}. This record does not assert independent reproduction. ### Directly Use Backbone ACCL: 54.31. Result ID: 6a44c5e773d097d845beb6d5. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen 3-8B (Thinking)"}. This record does not assert independent reproduction. ### Directly Use Backbone ACCL: 54.31. Result ID: 6a44c5e773d097d845beb6d8. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen 3-8B"}. This record does not assert independent reproduction. ### Qwen 2.5-3B ACCL: 51.67. Result ID: 6a44c5e773d097d845beb6c5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Family":"General Baselines (With Context)"}. This record does not assert independent reproduction. ### ASTUTE RAG ACCL: 50.28. Result ID: 6a44c5e773d097d845beb6da. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen 3-8B"}. This record does not assert independent reproduction. ### Directly Use Backbone ACCL: 49.58. Result ID: 6a44c5e773d097d845beb6cb. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Llama 3.2-3B"}. This record does not assert independent reproduction. ### GPT-5.1 (Thinking) ACCL: 49.44. Result ID: 6a44c5e773d097d845beb6c1. [Reporting paper]() Recorded evaluation configuration: {"Model Family":"Direct Generation (Without Context)"}. This record does not assert independent reproduction. ### Directly Use Backbone ACCL: 48.47. Result ID: 6a44c5e773d097d845beb6d0. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen 2.5-7B"}. This record does not assert independent reproduction. ### ASTUTE RAG ACCL: 43.75. Result ID: 6a44c5e773d097d845beb6d2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen 2.5-7B"}. This record does not assert independent reproduction. ### CAD Decoding ACCL: 40.28. Result ID: 6a44c5e773d097d845beb6d1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen 2.5-7B"}. This record does not assert independent reproduction. ### CAD Decoding ACCL: 40.28. Result ID: 6a44c5e773d097d845beb6d9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen 3-8B"}. This record does not assert independent reproduction. ### Qwen 2.5-1.5B ACCL: 39.86. Result ID: 6a44c5e773d097d845beb6c4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Family":"General Baselines (With Context)"}. This record does not assert independent reproduction. ### ASTUTE RAG ACCL: 39.72. Result ID: 6a44c5e773d097d845beb6cd. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama 3.2-3B"}. This record does not assert independent reproduction. ### CUAD (Default) ACCL: 34.31. Result ID: 6a44c5e773d097d845beb6c6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Family":"General Baselines (With Context)"}. This record does not assert independent reproduction. ### ADACAD (Default) ACCL: 34.31. Result ID: 6a44c5e773d097d845beb6c7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Family":"General Baselines (With Context)"}. This record does not assert independent reproduction. ### COCOA (Default) ACCL: 34.31. Result ID: 6a44c5e773d097d845beb6c8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Family":"General Baselines (With Context)"}. This record does not assert independent reproduction. ### GPT-4o-mini ACCL: 30.42. Result ID: 6a44c5e773d097d845beb6c0. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Family":"Direct Generation (Without Context)"}. This record does not assert independent reproduction. ### Llama 3.2-1B ACCL: 30. Result ID: 6a44c5e773d097d845beb6c2. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Family":"General Baselines (With Context)"}. This record does not assert independent reproduction. ### CAD Decoding ACCL: 29.72. Result ID: 6a44c5e773d097d845beb6cc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama 3.2-3B"}. This record does not assert independent reproduction. ### Llama 3.2-3B ACCL: 17.64. Result ID: 6a44c5e773d097d845beb6bd. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Family":"Direct Generation (Without Context)"}. This record does not assert independent reproduction. ### Qwen 3-8B (Thinking) ACCL: 17.36. Result ID: 6a44c5e773d097d845beb6bf. [Reporting paper]() Recorded evaluation configuration: {"Model Family":"Direct Generation (Without Context)"}. This record does not assert independent reproduction. ### Qwen 2.5-7B ACCL: 12.5. Result ID: 6a44c5e773d097d845beb6be. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Family":"Direct Generation (Without Context)"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.