Inter-Context Knowledge Conflict on PopQA
90.28ACCLKCR (“Graph”)
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| KCR (“Graph”)Backbone=Qwen 3-8B, Format=Graph2025.08 | 90.28 | 87.64 | 90 | |
| KCR (“Graph”)Backbone=Llama 3.2-3B, Format=Graph2025.08 | 87.5 | 83.61 | 87.78 | |
| KCR (“Text”)Backbone=Qwen 2.5-7B, Format=Text2025.08 | 86.67 | 82.5 | 86.53 | |
| KCR (“Text”)Backbone=Llama 3.2-3B, Format=Text2025.08 | 84.17 | 80.14 | 82.92 | |
| KCR (“Graph”)Backbone=Qwen 2.5-7B, Format=Graph2025.08 | 80.14 | 75.69 | 77.78 | |
| KCR (“Text”)Backbone=Qwen 3-8B, Format=Text2025.08 | 74.72 | 71.39 | 73.61 | |
| KCR (“Graph”)Backbone=Qwen 3-8B (Thinking), Format=Graph2025.08 | 69.58 | 65.56 | 69.03 | |
| KCR (“Text”)Backbone=Qwen 3-8B (Thinking), Format=Text2025.08 | 68.61 | 64.58 | 65.83 | |
| ASTUTE RAG (GPT-5.1)Model Family=General Baselines (With Context)2025.08 | 66.94 | 58.19 | 70.83 | |
| Llama 3.1-8BModel Family=General Baselines (With Context)2025.08 | 56.53 | 2.92 | 52.78 | |
| ASTUTE RAG (GPT-4o)Model Family=General Baselines (With Context)2025.08 | 54.86 | 50.56 | 60.97 | |
| Directly Use BackboneBackbone=Qwen 3-8B (Thinking)2025.08 | 54.31 | 50.56 | 54.72 | |
| Directly Use BackboneBackbone=Qwen 3-8B2025.08 | 54.31 | 50.56 | 55 | |
| Qwen 2.5-3BModel Family=General Baselines (With Context)2025.08 | 51.67 | 43.33 | 50.42 | |
| ASTUTE RAGBackbone=Qwen 3-8B2025.08 | 50.28 | 44.17 | 67.64 | |
| Directly Use BackboneBackbone=Llama 3.2-3B2025.08 | 49.58 | 33.75 | 49.44 | |
| GPT-5.1 (Thinking)Model Family=Direct Generation (Without Context)2025.08 | 49.44 | 13.89 | 48.19 | |
| Directly Use BackboneBackbone=Qwen 2.5-7B2025.08 | 48.47 | 41.81 | 47.78 | |
| ASTUTE RAGBackbone=Qwen 2.5-7B2025.08 | 43.75 | 38.89 | 45.56 | |
| CAD DecodingBackbone=Qwen 2.5-7B2025.08 | 40.28 | 17.5 | 43.61 | |
| CAD DecodingBackbone=Qwen 3-8B2025.08 | 40.28 | 17.5 | 43.61 | |
| Qwen 2.5-1.5BModel Family=General Baselines (With Context)2025.08 | 39.86 | 25.69 | 37.64 | |
| ASTUTE RAGBackbone=Llama 3.2-3B2025.08 | 39.72 | 2.08 | 74.58 | |
| CUAD (Default)Model Family=General Baselines (With Context)2025.08 | 34.31 | 29.58 | 30.56 | |
| ADACAD (Default)Model Family=General Baselines (With Context)2025.08 | 34.31 | 29.58 | 30.56 | |
| COCOA (Default)Model Family=General Baselines (With Context)2025.08 | 34.31 | 29.58 | 30.56 | |
| GPT-4o-miniModel Family=Direct Generation (Without Context)2025.08 | 30.42 | 24.31 | 25.42 | |
| Llama 3.2-1BModel Family=General Baselines (With Context)2025.08 | 30 | 1.39 | 26.25 | |
| CAD DecodingBackbone=Llama 3.2-3B2025.08 | 29.72 | 31.11 | 32.22 | |
| Llama 3.2-3BModel Family=Direct Generation (Without Context)2025.08 | 17.64 | 0.56 | 17.36 | |
| Qwen 3-8B (Thinking)Model Family=Direct Generation (Without Context)2025.08 | 17.36 | 13.06 | 14.44 | |
| Qwen 2.5-7BModel Family=Direct Generation (Without Context)2025.08 | 12.5 | 8.89 | 9.44 |