List your company
ResearchFormerly WizWand

Conversational Question Answering on COQA zero-shot (test)

70.85Exact Match (EM)Original
48.666854.425960.18565.9441Feb 9, 2026
Updated 8mo ago

Evaluation results

MethodLinks
OriginalModel=LLaMA3-70b2026.0270.85
CIS* (ours)Model=LLaMA3-70b, ρ̂=0.179, Theoretical Complexity (Comp*)=0.0114T2026.0270.68
DSModel=LLaMA3-70b, Theoretical Complexity (Comp*)=0.0625T2026.0270.45
CIS (ours)Model=LLaMA3-70b, ρ̂=0.183, Theoretical Complexity (Comp*)=0.0117T2026.0270.45
CIS (ours)Model=LLaMA3-70b, ρ̂=0.082, Theoretical Complexity (Comp*)=0.0052T2026.0270.21
Hshare-1Model=LLaMA3-70b, ρ̂=0.281, Theoretical Complexity (Comp*)=0.0180T2026.0270.1
CIS (ours)Model=LLaMA3-70b, ρ̂=0.064, Theoretical Complexity (Comp*)=0.0041T2026.0270.03
CIS* (ours)Model=LLaMA3-70b, ρ̂=0.079, Theoretical Complexity (Comp*)=0.0050T2026.0270.03
CIS* (ours)Model=LLaMA3-70b, ρ̂=0.061, Theoretical Complexity (Comp*)=0.0038T2026.0270.01
Hshare-2Model=LLaMA3-70b, ρ̂=0.125, Theoretical Complexity (Comp*)=0.0080T2026.0269.6
OriginalModel=Mistral-7b2026.0267.58
CIS* (ours)Model=Mistral-7b, ρ̂=0.068, Theoretical Complexity (Comp*)=0.0043T2026.0267.54
CIS* (ours)Model=Mistral-7b, ρ̂=0.091, Theoretical Complexity (Comp*)=0.0058T2026.0267.14
CIS (ours)Model=Mistral-7b, ρ̂=0.065, Theoretical Complexity (Comp*)=0.0041T2026.0267.1
CIS (ours)Model=Mistral-7b, ρ̂=0.194, Theoretical Complexity (Comp*)=0.0124T2026.0267.03
DSModel=Mistral-7b, Theoretical Complexity (Comp*)=0.0625T2026.0266.93
CIS* (ours)Model=Mistral-7b, ρ̂=0.189, Theoretical Complexity (Comp*)=0.0121T2026.0266.91
CIS (ours)Model=Mistral-7b, ρ̂=0.088, Theoretical Complexity (Comp*)=0.0056T2026.0266.84
Hshare-1Model=Mistral-7b, ρ̂=0.281, Theoretical Complexity (Comp*)=0.0180T2026.0265.6
Hshare-2Model=Mistral-7b, ρ̂=0.125, Theoretical Complexity (Comp*)=0.0080T2026.0263.87
CIS (ours)Model=LLaMA2-7b-chat, ρ̂=0.164, Theoretical Complexity (Comp*)=0.0104T2026.0260.85
CIS* (ours)Model=LLaMA2-7b-chat, ρ̂=0.063, Theoretical Complexity (Comp*)=0.0040T2026.0260.83
CIS (ours)Model=LLaMA2-7b-chat, ρ̂=0.087, Theoretical Complexity (Comp*)=0.0055T2026.0260.72
CIS (ours)Model=LLaMA2-7b-chat, ρ̂=0.062, Theoretical Complexity (Comp*)=0.0039T2026.0260.68
CIS* (ours)Model=LLaMA2-7b-chat, ρ̂=0.161, Theoretical Complexity (Comp*)=0.0103T2026.0260.63
CIS* (ours)Model=LLaMA2-7b-chat, ρ̂=0.084, Theoretical Complexity (Comp*)=0.0047T2026.0260.63
OriginalModel=LLaMA2-7b-chat2026.0259.97
DSModel=LLaMA2-7b-chat, Theoretical Complexity (Comp*)=0.0625T2026.0259.78
Hshare-1Model=LLaMA2-7b-chat, ρ̂=0.281, Theoretical Complexity (Comp*)=0.0180T2026.0258.98
QuestModel=LLaMA2-7b-chat, Theoretical Complexity (Comp*)=0.1250T2026.0257.13
Hshare-2Model=LLaMA2-7b-chat, ρ̂=0.125, Theoretical Complexity (Comp*)=0.0080T2026.0256.72
H2OModel=LLaMA2-7b-chat, Theoretical Complexity (Comp*)=O(1)2026.0249.52