Zero-shot Evaluation on LM Evaluation Harness Suite v1
79.71PIQADense
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| DensePrune Rate=0%, Model Family=LLaMA-3, Model Size=8B2024.06 | 79.71 | 60.19 | 72.61 | 80.09 | 50.34 | 68.59 | — | — | |
| PG pruningPrune Rate=30%, Model Family=LLaMA-3, Model Size=8B2024.06 | 72.25 | 43.56 | 59.04 | 59.85 | 29.44 | 52.83 | — | — | |
| LLM-PrunerPrune Rate=30%, Model Family=LLaMA-3, Model Size=8B2024.06 | 71.38 | 37.84 | 55.64 | 57.78 | 27.21 | 49.97 | — | — | |
| GPTData=Nemotron, Training Tokens=10B, Model Parameters=1.6B, num_fewshot=0, acc_norm=True (where applicable)2026.07 | 69.97 | 45.16 | 52.17 | 57.2 | 30.8 | 49.67 | 33 | 59.42 | |
| PRISMData=Nemotron, Training Tokens=50B, Model Parameters=1.6B, num_fewshot=0, acc_norm=True (where applicable)2026.07 | 69.75 | 45.97 | 53.75 | 57.37 | 30.46 | 49.75 | 32.6 | 58.35 | |
| PRISMData=Nemotron, Training Tokens=40B, Model Parameters=1.6B, num_fewshot=0, acc_norm=True (where applicable)2026.07 | 69.53 | 45.51 | 53.43 | 56.99 | 29.52 | 49.11 | 32.8 | 55.96 | |
| PRISMData=FineWeb, Training Tokens=20B, Model Parameters=1.6B, num_fewshot=0, acc_norm=True (where applicable)2026.07 | 69.48 | 44.73 | 53.75 | 56.61 | 31.14 | 49.94 | 34.2 | 59.69 | |
| PRISMData=Nemotron, Training Tokens=30B, Model Parameters=1.6B, num_fewshot=0, acc_norm=True (where applicable)2026.07 | 68.61 | 43.39 | 52.57 | 55.56 | 29.95 | 49.09 | 32.6 | 60.95 | |
| PRISMData=Nemotron, Training Tokens=20B, Model Parameters=1.6B, num_fewshot=0, acc_norm=True (where applicable)2026.07 | 68.44 | 41.58 | 51.78 | 52.06 | 29.35 | 46.74 | 30.8 | 53.18 | |
| SliceGPTPrune Rate=30%, Model Family=LLaMA-3, Model Size=8B2024.06 | 68.34 | 53.92 | 57.22 | 49.41 | 28.07 | 51.39 | — | — | |
| PRISMData=FineWeb, Training Tokens=10B, Model Parameters=1.6B, num_fewshot=0, acc_norm=True (where applicable)2026.07 | 68.12 | 41.86 | 52.49 | 54.88 | 30.38 | 48.95 | 35.2 | 59.69 | |
| GPTData=FineWeb, Training Tokens=10B, Model Parameters=1.6B, num_fewshot=0, acc_norm=True (where applicable)2026.07 | 67.95 | 42.65 | 51.93 | 54.84 | 29.69 | 48.89 | 34.2 | 60.95 | |
| PG pruningPrune Rate=40%, Model Family=LLaMA-3, Model Size=8B2024.06 | 67.63 | 37.36 | 56.91 | 50.67 | 24.91 | 47.5 | — | — | |
| PRISMData=Nemotron, Training Tokens=10B, Model Parameters=1.6B, num_fewshot=0, acc_norm=True (where applicable)2026.07 | 67.57 | 40.34 | 51.85 | 51.81 | 28.67 | 47.32 | 31.6 | 59.36 | |
| GPTData=FineWeb, Training Tokens=10B, Model Parameters=760M, num_fewshot=0, acc_norm=True (where applicable)2026.07 | 66.54 | 39.97 | 49.88 | 54.08 | 29.18 | 47.35 | 29.6 | 62.23 | |
| GPTData=FineWeb, Training Tokens=10B, Model Parameters=350M, num_fewshot=0, acc_norm=True (where applicable)2026.07 | 65.56 | 36.26 | 51.14 | 50.84 | 26.19 | 46.2 | 31.4 | 61.99 | |
| PRISMData=FineWeb, Training Tokens=10B, Model Parameters=760M, num_fewshot=0, acc_norm=True (where applicable)2026.07 | 65.29 | 36.46 | 50.59 | 50.72 | 27.47 | 45.22 | 32.8 | 53.24 | |
| PRISMData=FineWeb, Training Tokens=10B, Model Parameters=350M, num_fewshot=0, acc_norm=True (where applicable)2026.07 | 63.6 | 33.93 | 51.85 | 47.05 | 26.88 | 44.88 | 30.2 | 60.64 | |
| PG pruningPrune Rate=50%, Model Family=LLaMA-3, Model Size=8B2024.06 | 62.51 | 30.89 | 51.85 | 41.12 | 20.65 | 41.4 | — | — | |
| GPTData=FineWeb, Training Tokens=10B, Model Parameters=124M, num_fewshot=0, acc_norm=True (where applicable)2026.07 | 62.4 | 32.2 | 50.83 | 46.55 | 26.71 | 44.23 | 29.6 | 61.31 | |
| PRISMData=FineWeb, Training Tokens=10B, Model Parameters=124M, num_fewshot=0, acc_norm=True (where applicable)2026.07 | 61.15 | 31.27 | 50.51 | 44.65 | 24.57 | 42.91 | 29.8 | 58.41 |