List your company
ResearchFormerly WizWand

Inter-Context Knowledge Conflict on StrategyQA

83.2ACCLKCR (“Text”)
29.1243.1657.271.24Aug 2, 2025
Updated 4mo ago

Evaluation results

MethodLinks
KCR (“Text”)Backbone=Qwen 2.5-7B, Format=Text2025.0883.279.279.2
KCR (“Graph”)Backbone=Llama 3.2-3B, Format=Graph2025.0882.477.677.6
KCR (“Text”)Backbone=Llama 3.2-3B, Format=Text2025.0877.676.876.8
Qwen 3-8B (Thinking)Model Family=Direct Generation (Without Context)2025.08767276.8
KCR (“Graph”)Backbone=Qwen 2.5-7B, Format=Graph2025.0875.267.267.2
GPT-4o-miniModel Family=Direct Generation (Without Context)2025.0874.474.474.4
Qwen 2.5-7BModel Family=Direct Generation (Without Context)2025.0868.867.267.2
GPT-5.1 (Thinking)Model Family=Direct Generation (Without Context)2025.0866.4067.2
KCR (“Text”)Backbone=Qwen 3-8B (Thinking), Format=Text2025.08646464
ASTUTE RAG (GPT-4o)Model Family=General Baselines (With Context)2025.0859.257.658.4
Directly Use BackboneBackbone=Qwen 2.5-7B2025.0859.259.259.2
KCR (“Text”)Backbone=Qwen 3-8B, Format=Text2025.0858.6154.4458.06
ASTUTE RAG (GPT-5.1)Model Family=General Baselines (With Context)2025.0857.621.659.2
Directly Use BackboneBackbone=Qwen 3-8B (Thinking)2025.08565656
KCR (“Graph”)Backbone=Qwen 3-8B (Thinking), Format=Graph2025.08565656
Qwen 2.5-3BModel Family=General Baselines (With Context)2025.0854.453.653.6
Directly Use BackboneBackbone=Qwen 3-8B2025.0853.653.653.6
KCR (“Graph”)Backbone=Qwen 3-8B, Format=Graph2025.0852.7848.8953.19
Directly Use BackboneBackbone=Llama 3.2-3B2025.0851.239.242.4
Qwen 2.5-1.5BModel Family=General Baselines (With Context)2025.0850.438.439.2
CUAD (Default)Model Family=General Baselines (With Context)2025.0850.450.450.4
ADACAD (Default)Model Family=General Baselines (With Context)2025.0850.450.450.4
COCOA (Default)Model Family=General Baselines (With Context)2025.0850.450.450.4
ASTUTE RAGBackbone=Qwen 3-8B2025.0850.448.870.4
Llama 3.2-3BModel Family=Direct Generation (Without Context)2025.0846.4032
CAD DecodingBackbone=Llama 3.2-3B2025.0846.432.848.8
ASTUTE RAGBackbone=Llama 3.2-3B2025.0846.438.443.2
Llama 3.2-1BModel Family=General Baselines (With Context)2025.0844.81.66.4
Llama 3.1-8BModel Family=General Baselines (With Context)2025.0844.8040
ASTUTE RAGBackbone=Qwen 2.5-7B2025.0836.83636
CAD DecodingBackbone=Qwen 2.5-7B2025.0831.2034.4
CAD DecodingBackbone=Qwen 3-8B2025.0831.2034.4