Open Book Question Answering on OpenBookQA
89.4Normalized Log AccuracyT-Free
Evaluation results
| Method | Links | ||
|---|---|---|---|
| T-Freeshots=10-shot2026.03 | 89.4 | 4.85 | |
| HATifiedshots=10-shot2026.03 | 86.8 | 4.85 | |
| Llamashots=10-shot2026.03 | 84.6 | 4.35 | |
| No pruningSparsity=0, Backbone=Gemma-2 9B, Evaluation Protocol=Zero-shot2026.03 | 47.2 | — | |
| Magnitude-DimSparsity=10%, Backbone=Gemma-2 9B, Evaluation Protocol=Zero-shot2026.03 | 43.2 | — | |
| DIETSparsity=10%, Backbone=Gemma-2 9B, Evaluation Protocol=Zero-shot2026.03 | 40 | — | |
| Magnitude-DimSparsity=20%, Backbone=Gemma-2 9B, Evaluation Protocol=Zero-shot2026.03 | 31.6 | — | |
| DIETSparsity=20%, Backbone=Gemma-2 9B, Evaluation Protocol=Zero-shot2026.03 | 31.2 | — | |
| PuDDingSparsity=10%, Backbone=Gemma-2 9B, Evaluation Protocol=Zero-shot2026.03 | 30.2 | — | |
| PuDDingSparsity=20%, Backbone=Gemma-2 9B, Evaluation Protocol=Zero-shot2026.03 | 29.2 | — | |
| SliceGPTSparsity=20%, Backbone=Gemma-2 9B, Evaluation Protocol=Zero-shot2026.03 | 27.6 | — | |
| SliceGPTSparsity=10%, Backbone=Gemma-2 9B, Evaluation Protocol=Zero-shot2026.03 | 27.4 | — |