List your company
ResearchFormerly WizWand

Inter-Context Knowledge Conflict on PopQA

90.28ACCLKCR (“Graph”)
9.388830.389451.3972.3906Aug 2, 2025
Updated 4mo ago

Evaluation results

MethodLinks
KCR (“Graph”)Backbone=Qwen 3-8B, Format=Graph2025.0890.2887.6490
KCR (“Graph”)Backbone=Llama 3.2-3B, Format=Graph2025.0887.583.6187.78
KCR (“Text”)Backbone=Qwen 2.5-7B, Format=Text2025.0886.6782.586.53
KCR (“Text”)Backbone=Llama 3.2-3B, Format=Text2025.0884.1780.1482.92
KCR (“Graph”)Backbone=Qwen 2.5-7B, Format=Graph2025.0880.1475.6977.78
KCR (“Text”)Backbone=Qwen 3-8B, Format=Text2025.0874.7271.3973.61
KCR (“Graph”)Backbone=Qwen 3-8B (Thinking), Format=Graph2025.0869.5865.5669.03
KCR (“Text”)Backbone=Qwen 3-8B (Thinking), Format=Text2025.0868.6164.5865.83
ASTUTE RAG (GPT-5.1)Model Family=General Baselines (With Context)2025.0866.9458.1970.83
Llama 3.1-8BModel Family=General Baselines (With Context)2025.0856.532.9252.78
ASTUTE RAG (GPT-4o)Model Family=General Baselines (With Context)2025.0854.8650.5660.97
Directly Use BackboneBackbone=Qwen 3-8B (Thinking)2025.0854.3150.5654.72
Directly Use BackboneBackbone=Qwen 3-8B2025.0854.3150.5655
Qwen 2.5-3BModel Family=General Baselines (With Context)2025.0851.6743.3350.42
ASTUTE RAGBackbone=Qwen 3-8B2025.0850.2844.1767.64
Directly Use BackboneBackbone=Llama 3.2-3B2025.0849.5833.7549.44
GPT-5.1 (Thinking)Model Family=Direct Generation (Without Context)2025.0849.4413.8948.19
Directly Use BackboneBackbone=Qwen 2.5-7B2025.0848.4741.8147.78
ASTUTE RAGBackbone=Qwen 2.5-7B2025.0843.7538.8945.56
CAD DecodingBackbone=Qwen 2.5-7B2025.0840.2817.543.61
CAD DecodingBackbone=Qwen 3-8B2025.0840.2817.543.61
Qwen 2.5-1.5BModel Family=General Baselines (With Context)2025.0839.8625.6937.64
ASTUTE RAGBackbone=Llama 3.2-3B2025.0839.722.0874.58
CUAD (Default)Model Family=General Baselines (With Context)2025.0834.3129.5830.56
ADACAD (Default)Model Family=General Baselines (With Context)2025.0834.3129.5830.56
COCOA (Default)Model Family=General Baselines (With Context)2025.0834.3129.5830.56
GPT-4o-miniModel Family=Direct Generation (Without Context)2025.0830.4224.3125.42
Llama 3.2-1BModel Family=General Baselines (With Context)2025.08301.3926.25
CAD DecodingBackbone=Llama 3.2-3B2025.0829.7231.1132.22
Llama 3.2-3BModel Family=Direct Generation (Without Context)2025.0817.640.5617.36
Qwen 3-8B (Thinking)Model Family=Direct Generation (Without Context)2025.0817.3613.0614.44
Qwen 2.5-7BModel Family=Direct Generation (Without Context)2025.0812.58.899.44