Zero-shot Reasoning on OpenBookQA
44AccuracyLlama2-7B
Evaluation results
| Method | Links | |
|---|---|---|
| Llama2-7BBackbone=Llama2-7B2026.02 | 44 | |
| BaselineRatio=1.0, Memory Footprint (GB)=12.6GB, Zero-shot=true2026.04 | 34 | |
| Dobi-SVDBackbone=Llama2-7B, Training-free=false, CR=0.22026.02 | 32.8 | |
| SVD-LLM(W)Ratio=0.8, Memory Footprint (GB)=10.1 GB, Zero-shot=true2026.04 | 31 | |
| COMPOTBackbone=Llama2-7B, Training-free=true, CR=0.22026.02 | 30.4 | |
| Dobi-SVD(w)Ratio=0.8, Memory Footprint (GB)=10.1 GB, Zero-shot=true2026.04 | 30 | |
| Dobi-SVDBackbone=Llama2-7B, Training-free=false, CR=0.42026.02 | 29.2 | |
| ASVDRatio=0.8, Memory Footprint (GB)=10.1 GB, Zero-shot=true2026.04 | 29 | |
| Swift-SVD*Ratio=0.8, Memory Footprint (GB)=10.1 GB, Zero-shot=true2026.04 | 27 | |
| Dobi-SVDBackbone=Llama2-7B, Training-free=false, CR=0.62026.02 | 26 | |
| Swift-SVDRatio=0.8, Memory Footprint (GB)=10.1 GB, Zero-shot=true2026.04 | 26 | |
| COMPOTBackbone=Llama2-7B, Training-free=true, CR=0.42026.02 | 25.2 | |
| SVD-LLM(W)Ratio=0.6, Memory Footprint (GB)=7.7 GB, Zero-shot=true2026.04 | 25 | |
| Swift-SVD*Ratio=0.6, Memory Footprint (GB)=7.7 GB, Zero-shot=true2026.04 | 23 | |
| Dobi-SVD(w)Ratio=0.6, Memory Footprint (GB)=7.7 GB, Zero-shot=true2026.04 | 22 | |
| Swift-SVDRatio=0.6, Memory Footprint (GB)=7.7 GB, Zero-shot=true2026.04 | 21 | |
| Swift-SVDRatio=0.4, Memory Footprint (GB)=5.3 GB, Zero-shot=true2026.04 | 16 | |
| Swift-SVD*Ratio=0.4, Memory Footprint (GB)=5.3 GB, Zero-shot=true2026.04 | 16 | |
| Dobi-SVD(w)Ratio=0.4, Memory Footprint (GB)=5.3 GB, Zero-shot=true2026.04 | 14 | |
| COMPOTBackbone=Llama2-7B, Training-free=true, CR=0.62026.02 | 13.4 | |
| SVD-LLM(W)Ratio=0.4, Memory Footprint (GB)=5.3 GB, Zero-shot=true2026.04 | 10 | |
| FWSVDRatio=0.8, Memory Footprint (GB)=10.1 GB, Zero-shot=true2026.04 | 9 | |
| ASVDRatio=0.6, Memory Footprint (GB)=7.7 GB, Zero-shot=true2026.04 | 8 | |
| FWSVDRatio=0.6, Memory Footprint (GB)=7.7 GB, Zero-shot=true2026.04 | 6 | |
| FWSVDRatio=0.4, Memory Footprint (GB)=5.3 GB, Zero-shot=true2026.04 | 6 | |
| ASVDRatio=0.4, Memory Footprint (GB)=5.3 GB, Zero-shot=true2026.04 | 5 |