List your company
ResearchFormerly WizWand

Conversational Question Answering on CoQA (Accuracy)

75.9AccuracyLlama-2
-0.95618.99738.9558.903Oct 24, 2024Jan 30, 2025May 8, 2025Aug 15, 2025Nov 21, 2025Feb 27, 2026Jun 6, 2026
Updated 4mo ago

Evaluation results

MethodLinks
Llama-2#Param=6.9B, Training Tokens=2T, Shots=zero-shot2024.1075.9
Base modelBackbone=Llama3-8B-Instruct2026.0175.8
Circuit BreakBackbone=Llama3-8B-Instruct2026.0175.7
CKUBackbone=Llama3-8B-Instruct2026.0175.7
JPUBackbone=Llama3-8B-Instruct2026.0175.7
EraserBackbone=Llama3-8B-Instruct2026.0175.48
Safe UnlearningBackbone=Llama3-8B-Instruct2026.0175.26
RSFTBackbone=Llama3-8B-Instruct2026.0174.85
Read-ME#Param=4.7B-17B, Training Tokens=1B, Shots=zero-shot2024.1074.8
Sheared-Llama#Param=2.7B, Training Tokens=50B, Shots=zero-shot2024.1071.7
DenseCompression Ratio=0%, Post-training compensation=No, Evaluation Protocol=Zero-shot2024.1266.7
Open-Llama-v2#Param=6.9B, Training Tokens=1T, Shots=zero-shot2024.1064.5
Pythia#Param=6.9B, Training Tokens=300B, Shots=zero-shot2024.1063.6
GRASPCompression Ratio=25%, Post-training compensation=Yes, Evaluation Protocol=Zero-shot2024.1263.2
Pythia#Param=2.8B, Training Tokens=300B, Shots=zero-shot2024.1061.9
LLM-Streamline-FFNCompression Ratio=25%, Post-training compensation=Yes, Evaluation Protocol=Zero-shot2024.1260.6
LLM-Streamline-LayerCompression Ratio=25%, Post-training compensation=Yes, Evaluation Protocol=Zero-shot2024.1259.2
CKUBackbone=Llama2-7B-Chat2026.0159.01
JPUBackbone=Llama2-7B-Chat2026.0158.92
Base modelBackbone=Llama2-7B-Chat2026.0158.67
Circuit BreakBackbone=Llama2-7B-Chat2026.0158.65
EraserBackbone=Llama2-7B-Chat2026.0158.61
Safe UnlearningBackbone=Llama2-7B-Chat2026.0158.55
RSFTBackbone=Llama2-7B-Chat2026.0157.4
Open-Llama-v2#Param=3.4B, Training Tokens=1T, Shots=zero-shot2024.1054.4
ShortGPTCompression Ratio=25%, Post-training compensation=Yes, Evaluation Protocol=Zero-shot2024.1251.7
SliceGPTCompression Ratio=25%, Post-training compensation=Yes, Evaluation Protocol=Zero-shot2024.1249.6
LLMPrunerCompression Ratio=25%, Post-training compensation=Yes, Evaluation Protocol=Zero-shot2024.1248.5
LaCoCompression Ratio=25%, Post-training compensation=Yes, Evaluation Protocol=Zero-shot2024.1245.7
TN-gramLayers (L)=182026.068.92
EngramLayers (L)=182026.068.18
Raw GPTLayers (L)=182026.067.95
TN-gramLayers (L)=92026.065.93
EngramLayers (L)=92026.065.45
Raw GPTLayers (L)=92026.062