Inter-Context Knowledge Conflict on StrategyQA
83.2ACCLKCR (“Text”)
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| KCR (“Text”)Backbone=Qwen 2.5-7B, Format=Text2025.08 | 83.2 | 79.2 | 79.2 | |
| KCR (“Graph”)Backbone=Llama 3.2-3B, Format=Graph2025.08 | 82.4 | 77.6 | 77.6 | |
| KCR (“Text”)Backbone=Llama 3.2-3B, Format=Text2025.08 | 77.6 | 76.8 | 76.8 | |
| Qwen 3-8B (Thinking)Model Family=Direct Generation (Without Context)2025.08 | 76 | 72 | 76.8 | |
| KCR (“Graph”)Backbone=Qwen 2.5-7B, Format=Graph2025.08 | 75.2 | 67.2 | 67.2 | |
| GPT-4o-miniModel Family=Direct Generation (Without Context)2025.08 | 74.4 | 74.4 | 74.4 | |
| Qwen 2.5-7BModel Family=Direct Generation (Without Context)2025.08 | 68.8 | 67.2 | 67.2 | |
| GPT-5.1 (Thinking)Model Family=Direct Generation (Without Context)2025.08 | 66.4 | 0 | 67.2 | |
| KCR (“Text”)Backbone=Qwen 3-8B (Thinking), Format=Text2025.08 | 64 | 64 | 64 | |
| ASTUTE RAG (GPT-4o)Model Family=General Baselines (With Context)2025.08 | 59.2 | 57.6 | 58.4 | |
| Directly Use BackboneBackbone=Qwen 2.5-7B2025.08 | 59.2 | 59.2 | 59.2 | |
| KCR (“Text”)Backbone=Qwen 3-8B, Format=Text2025.08 | 58.61 | 54.44 | 58.06 | |
| ASTUTE RAG (GPT-5.1)Model Family=General Baselines (With Context)2025.08 | 57.6 | 21.6 | 59.2 | |
| Directly Use BackboneBackbone=Qwen 3-8B (Thinking)2025.08 | 56 | 56 | 56 | |
| KCR (“Graph”)Backbone=Qwen 3-8B (Thinking), Format=Graph2025.08 | 56 | 56 | 56 | |
| Qwen 2.5-3BModel Family=General Baselines (With Context)2025.08 | 54.4 | 53.6 | 53.6 | |
| Directly Use BackboneBackbone=Qwen 3-8B2025.08 | 53.6 | 53.6 | 53.6 | |
| KCR (“Graph”)Backbone=Qwen 3-8B, Format=Graph2025.08 | 52.78 | 48.89 | 53.19 | |
| Directly Use BackboneBackbone=Llama 3.2-3B2025.08 | 51.2 | 39.2 | 42.4 | |
| Qwen 2.5-1.5BModel Family=General Baselines (With Context)2025.08 | 50.4 | 38.4 | 39.2 | |
| CUAD (Default)Model Family=General Baselines (With Context)2025.08 | 50.4 | 50.4 | 50.4 | |
| ADACAD (Default)Model Family=General Baselines (With Context)2025.08 | 50.4 | 50.4 | 50.4 | |
| COCOA (Default)Model Family=General Baselines (With Context)2025.08 | 50.4 | 50.4 | 50.4 | |
| ASTUTE RAGBackbone=Qwen 3-8B2025.08 | 50.4 | 48.8 | 70.4 | |
| Llama 3.2-3BModel Family=Direct Generation (Without Context)2025.08 | 46.4 | 0 | 32 | |
| CAD DecodingBackbone=Llama 3.2-3B2025.08 | 46.4 | 32.8 | 48.8 | |
| ASTUTE RAGBackbone=Llama 3.2-3B2025.08 | 46.4 | 38.4 | 43.2 | |
| Llama 3.2-1BModel Family=General Baselines (With Context)2025.08 | 44.8 | 1.6 | 6.4 | |
| Llama 3.1-8BModel Family=General Baselines (With Context)2025.08 | 44.8 | 0 | 40 | |
| ASTUTE RAGBackbone=Qwen 2.5-7B2025.08 | 36.8 | 36 | 36 | |
| CAD DecodingBackbone=Qwen 2.5-7B2025.08 | 31.2 | 0 | 34.4 | |
| CAD DecodingBackbone=Qwen 3-8B2025.08 | 31.2 | 0 | 34.4 |