Language Understanding and Reasoning on LUE Suite (Zyda2 calibration test)
58.28ARC-cMoonlight
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| MoonlightPruning ratio=0%, Evaluation protocol=Zero-shot, Calibration samples=100, Backbone=Moonlight2025.07 | 58.28 | 82.49 | 80.4 | 92 | 67.3 | 45.6 | 81.12 | 65.7 | 71.11 | 71.56 | |
| MoNEPruning ratio=25%, Evaluation protocol=Zero-shot, Calibration samples=100, Backbone=Moonlight2025.07 | 55.89 | 80.6 | 79.57 | 90 | 55.23 | 46.8 | 80.85 | 61.01 | 71.98 | 69.1 | |
| RSPruning ratio=25%, Evaluation protocol=Zero-shot, Calibration samples=100, Backbone=Moonlight2025.07 | 55.8 | 80.64 | 78.69 | 90 | 46.73 | 46.4 | 81.01 | 58.84 | 72.3 | 67.82 | |
| FLAPPruning ratio=25%, Evaluation protocol=Zero-shot, Calibration samples=100, Backbone=Moonlight2025.07 | 48.55 | 76.01 | 75.93 | 90 | 55.84 | 42.2 | 77.97 | 64.26 | 68.19 | 66.55 | |
| MC-SMoEPruning ratio=25%, Evaluation protocol=Zero-shot, Calibration samples=100, Backbone=Moonlight2025.07 | 47.61 | 73.15 | 78.72 | 89 | 46.11 | 43.6 | 80.36 | 56.32 | 71.43 | 65.14 | |
| AngularPruning ratio=25%, Evaluation protocol=Zero-shot, Calibration samples=100, Backbone=Moonlight2025.07 | 39.76 | 52.69 | 38.9 | 79 | 42.57 | 32.2 | 68.5 | 61.01 | 62.04 | 52.96 |