Conversational Question Answering on CoQA (Accuracy)
75.9AccuracyLlama-2
Evaluation Results
| Method | Links | |
|---|---|---|
| Llama-2#Param=6.9B, Training Tokens=2T, Shots=zero-shot2024.10 | 75.9 | |
| Base modelBackbone=Llama3-8B-Instruct2026.01 | 75.8 | |
| Circuit BreakBackbone=Llama3-8B-Instruct2026.01 | 75.7 | |
| CKUBackbone=Llama3-8B-Instruct2026.01 | 75.7 | |
| JPUBackbone=Llama3-8B-Instruct2026.01 | 75.7 | |
| EraserBackbone=Llama3-8B-Instruct2026.01 | 75.48 | |
| Safe UnlearningBackbone=Llama3-8B-Instruct2026.01 | 75.26 | |
| RSFTBackbone=Llama3-8B-Instruct2026.01 | 74.85 | |
| Read-ME#Param=4.7B-17B, Training Tokens=1B, Shots=zero-shot2024.10 | 74.8 | |
| Sheared-Llama#Param=2.7B, Training Tokens=50B, Shots=zero-shot2024.10 | 71.7 | |
| DenseCompression Ratio=0%, Post-training compensation=No, Evaluation Protocol=Zero-shot2024.12 | 66.7 | |
| Open-Llama-v2#Param=6.9B, Training Tokens=1T, Shots=zero-shot2024.10 | 64.5 | |
| Pythia#Param=6.9B, Training Tokens=300B, Shots=zero-shot2024.10 | 63.6 | |
| GRASPCompression Ratio=25%, Post-training compensation=Yes, Evaluation Protocol=Zero-shot2024.12 | 63.2 | |
| Pythia#Param=2.8B, Training Tokens=300B, Shots=zero-shot2024.10 | 61.9 | |
| LLM-Streamline-FFNCompression Ratio=25%, Post-training compensation=Yes, Evaluation Protocol=Zero-shot2024.12 | 60.6 | |
| LLM-Streamline-LayerCompression Ratio=25%, Post-training compensation=Yes, Evaluation Protocol=Zero-shot2024.12 | 59.2 | |
| CKUBackbone=Llama2-7B-Chat2026.01 | 59.01 | |
| JPUBackbone=Llama2-7B-Chat2026.01 | 58.92 | |
| Base modelBackbone=Llama2-7B-Chat2026.01 | 58.67 | |
| Circuit BreakBackbone=Llama2-7B-Chat2026.01 | 58.65 | |
| EraserBackbone=Llama2-7B-Chat2026.01 | 58.61 | |
| Safe UnlearningBackbone=Llama2-7B-Chat2026.01 | 58.55 | |
| RSFTBackbone=Llama2-7B-Chat2026.01 | 57.4 | |
| Open-Llama-v2#Param=3.4B, Training Tokens=1T, Shots=zero-shot2024.10 | 54.4 | |
| ShortGPTCompression Ratio=25%, Post-training compensation=Yes, Evaluation Protocol=Zero-shot2024.12 | 51.7 | |
| SliceGPTCompression Ratio=25%, Post-training compensation=Yes, Evaluation Protocol=Zero-shot2024.12 | 49.6 | |
| LLMPrunerCompression Ratio=25%, Post-training compensation=Yes, Evaluation Protocol=Zero-shot2024.12 | 48.5 | |
| LaCoCompression Ratio=25%, Post-training compensation=Yes, Evaluation Protocol=Zero-shot2024.12 | 45.7 | |
| TN-gramLayers (L)=182026.06 | 8.92 | |
| EngramLayers (L)=182026.06 | 8.18 | |
| Raw GPTLayers (L)=182026.06 | 7.95 | |
| TN-gramLayers (L)=92026.06 | 5.93 | |
| EngramLayers (L)=92026.06 | 5.45 | |
| Raw GPTLayers (L)=92026.06 | 2 |