Question Answering on OpenBookQA 2018
43Acc_normBaseline
Evaluation results
| Method | Links | |
|---|---|---|
| BaselineBackbone=Llama 3.1 8B Instruct, Compression Ratio=1.00x, Zero-shot=true2026.06 | 43 | |
| SigmaScaleBackbone=Llama 3.1 8B Instruct, Compression Ratio=0.90x, Zero-shot=true2026.06 | 43 | |
| SVD-LLMBackbone=Qwen3-8B, Compression Ratio=0.90x, Zero-shot=true2026.06 | 43 | |
| SigmaScale (KD)Backbone=Llama 3.1 8B Instruct, Compression Ratio=0.90x, Zero-shot=true2026.06 | 42.8 | |
| ASVD+Backbone=Llama 3.1 8B Instruct, Compression Ratio=0.90x, Zero-shot=true2026.06 | 42.6 | |
| BaselineBackbone=Qwen3-8B, Compression Ratio=1.00x, Zero-shot=true2026.06 | 41.4 | |
| SigmaScaleBackbone=Qwen3-8B, Compression Ratio=0.90x, Zero-shot=true2026.06 | 40.8 | |
| SVD-LLMBackbone=Qwen3-8B, Compression Ratio=0.75x, Zero-shot=true2026.06 | 40.8 | |
| SigmaScale (KD)Backbone=Qwen3-8B, Compression Ratio=0.90x, Zero-shot=true2026.06 | 40.4 | |
| SigmaScaleBackbone=Qwen3-8B, Compression Ratio=0.75x, Zero-shot=true2026.06 | 40.4 | |
| SigmaScale (KD)Backbone=Qwen3-8B, Compression Ratio=0.75x, Zero-shot=true2026.06 | 40 | |
| SVD-LLMBackbone=Llama 3.1 8B Instruct, Compression Ratio=0.90x, Zero-shot=true2026.06 | 37.8 | |
| SigmaScale (KD)Backbone=Llama 3.1 8B Instruct, Compression Ratio=0.75x, Zero-shot=true2026.06 | 37 | |
| SigmaScaleBackbone=Llama 3.1 8B Instruct, Compression Ratio=0.75x, Zero-shot=true2026.06 | 36.8 | |
| SVD-LLMBackbone=Qwen3-8B, Compression Ratio=0.50x, Zero-shot=true2026.06 | 36.8 | |
| ASVD+Backbone=Qwen3-8B, Compression Ratio=0.75x, Zero-shot=true2026.06 | 36.2 | |
| ASVD+Backbone=Qwen3-8B, Compression Ratio=0.90x, Zero-shot=true2026.06 | 36 | |
| ASVD+Backbone=Llama 3.1 8B Instruct, Compression Ratio=0.75x, Zero-shot=true2026.06 | 33.4 | |
| SVD-LLMBackbone=Llama 3.1 8B Instruct, Compression Ratio=0.75x, Zero-shot=true2026.06 | 32.6 | |
| SigmaScale (KD)Backbone=Qwen3-8B, Compression Ratio=0.50x, Zero-shot=true2026.06 | 32.6 | |
| SigmaScaleBackbone=Qwen3-8B, Compression Ratio=0.50x, Zero-shot=true2026.06 | 32 | |
| ASVD+Backbone=Llama 3.1 8B Instruct, Compression Ratio=0.50x, Zero-shot=true2026.06 | 30 | |
| ASVD+Backbone=Qwen3-8B, Compression Ratio=0.50x, Zero-shot=true2026.06 | 29.8 | |
| SigmaScale (KD)Backbone=Llama 3.1 8B Instruct, Compression Ratio=0.50x, Zero-shot=true2026.06 | 28 | |
| SVD-LLMBackbone=Llama 3.1 8B Instruct, Compression Ratio=0.50x, Zero-shot=true2026.06 | 27.2 | |
| SigmaScaleBackbone=Llama 3.1 8B Instruct, Compression Ratio=0.50x, Zero-shot=true2026.06 | 26.8 |