Conversational Question Answering on COQA zero-shot (test)
70.85Exact Match (EM)Original
Evaluation Results
| Method | Links | |
|---|---|---|
| OriginalModel=LLaMA3-70b2026.02 | 70.85 | |
| CIS* (ours)Model=LLaMA3-70b, ρ̂=0.179, Theoretical Complexity (Comp*)=0.0114T2026.02 | 70.68 | |
| DSModel=LLaMA3-70b, Theoretical Complexity (Comp*)=0.0625T2026.02 | 70.45 | |
| CIS (ours)Model=LLaMA3-70b, ρ̂=0.183, Theoretical Complexity (Comp*)=0.0117T2026.02 | 70.45 | |
| CIS (ours)Model=LLaMA3-70b, ρ̂=0.082, Theoretical Complexity (Comp*)=0.0052T2026.02 | 70.21 | |
| Hshare-1Model=LLaMA3-70b, ρ̂=0.281, Theoretical Complexity (Comp*)=0.0180T2026.02 | 70.1 | |
| CIS (ours)Model=LLaMA3-70b, ρ̂=0.064, Theoretical Complexity (Comp*)=0.0041T2026.02 | 70.03 | |
| CIS* (ours)Model=LLaMA3-70b, ρ̂=0.079, Theoretical Complexity (Comp*)=0.0050T2026.02 | 70.03 | |
| CIS* (ours)Model=LLaMA3-70b, ρ̂=0.061, Theoretical Complexity (Comp*)=0.0038T2026.02 | 70.01 | |
| Hshare-2Model=LLaMA3-70b, ρ̂=0.125, Theoretical Complexity (Comp*)=0.0080T2026.02 | 69.6 | |
| OriginalModel=Mistral-7b2026.02 | 67.58 | |
| CIS* (ours)Model=Mistral-7b, ρ̂=0.068, Theoretical Complexity (Comp*)=0.0043T2026.02 | 67.54 | |
| CIS* (ours)Model=Mistral-7b, ρ̂=0.091, Theoretical Complexity (Comp*)=0.0058T2026.02 | 67.14 | |
| CIS (ours)Model=Mistral-7b, ρ̂=0.065, Theoretical Complexity (Comp*)=0.0041T2026.02 | 67.1 | |
| CIS (ours)Model=Mistral-7b, ρ̂=0.194, Theoretical Complexity (Comp*)=0.0124T2026.02 | 67.03 | |
| DSModel=Mistral-7b, Theoretical Complexity (Comp*)=0.0625T2026.02 | 66.93 | |
| CIS* (ours)Model=Mistral-7b, ρ̂=0.189, Theoretical Complexity (Comp*)=0.0121T2026.02 | 66.91 | |
| CIS (ours)Model=Mistral-7b, ρ̂=0.088, Theoretical Complexity (Comp*)=0.0056T2026.02 | 66.84 | |
| Hshare-1Model=Mistral-7b, ρ̂=0.281, Theoretical Complexity (Comp*)=0.0180T2026.02 | 65.6 | |
| Hshare-2Model=Mistral-7b, ρ̂=0.125, Theoretical Complexity (Comp*)=0.0080T2026.02 | 63.87 | |
| CIS (ours)Model=LLaMA2-7b-chat, ρ̂=0.164, Theoretical Complexity (Comp*)=0.0104T2026.02 | 60.85 | |
| CIS* (ours)Model=LLaMA2-7b-chat, ρ̂=0.063, Theoretical Complexity (Comp*)=0.0040T2026.02 | 60.83 | |
| CIS (ours)Model=LLaMA2-7b-chat, ρ̂=0.087, Theoretical Complexity (Comp*)=0.0055T2026.02 | 60.72 | |
| CIS (ours)Model=LLaMA2-7b-chat, ρ̂=0.062, Theoretical Complexity (Comp*)=0.0039T2026.02 | 60.68 | |
| CIS* (ours)Model=LLaMA2-7b-chat, ρ̂=0.161, Theoretical Complexity (Comp*)=0.0103T2026.02 | 60.63 | |
| CIS* (ours)Model=LLaMA2-7b-chat, ρ̂=0.084, Theoretical Complexity (Comp*)=0.0047T2026.02 | 60.63 | |
| OriginalModel=LLaMA2-7b-chat2026.02 | 59.97 | |
| DSModel=LLaMA2-7b-chat, Theoretical Complexity (Comp*)=0.0625T2026.02 | 59.78 | |
| Hshare-1Model=LLaMA2-7b-chat, ρ̂=0.281, Theoretical Complexity (Comp*)=0.0180T2026.02 | 58.98 | |
| QuestModel=LLaMA2-7b-chat, Theoretical Complexity (Comp*)=0.1250T2026.02 | 57.13 | |
| Hshare-2Model=LLaMA2-7b-chat, ρ̂=0.125, Theoretical Complexity (Comp*)=0.0080T2026.02 | 56.72 | |
| H2OModel=LLaMA2-7b-chat, Theoretical Complexity (Comp*)=O(1)2026.02 | 49.52 |