Zero-shot Reasoning and Question Answering on (ARC-C, ARC-E, H'SWAG, LAM'DA, OPENBKQA, PIQA, W'GRANDE)
32.76ARC-C ScoreASVD
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| ASVDOptimizer=NUMUON, Compression Rate=40%, Backbone Model=Llama3-1.8B2026.03 | 32.76 | 62.88 | 50.27 | 42.64 | 37.2 | 71.38 | 56.2 | 50.48 | |
| SVD-LLMOptimizer=NUMUON, Compression Rate=40%, Backbone Model=Llama3-1.8B2026.03 | 31.66 | 60.77 | 49.49 | 38.87 | 34.8 | 69.37 | 57.38 | 48.91 | |
| Dobi-SVDOptimizer=NUMUON, Compression Rate=40%, Backbone Model=Llama3-1.8B2026.03 | 31.31 | 60.23 | 51.98 | 39.59 | 34.8 | 70.18 | 57.54 | 49.38 | |
| Dobi-SVDOptimizer=ADAMW, Compression Rate=40%, Backbone Model=Llama3-1.8B2026.03 | 30.72 | 49.79 | 43.15 | 31.3 | 33.2 | 64.69 | 53.67 | 43.79 | |
| SVD-LLMOptimizer=ADAMW, Compression Rate=40%, Backbone Model=Llama3-1.8B2026.03 | 28.41 | 49.37 | 40.69 | 29.52 | 30 | 65.13 | 51.22 | 42.05 | |
| Dobi-SVDOptimizer=MUON, Compression Rate=40%, Backbone Model=Llama3-1.8B2026.03 | 28.16 | 52.27 | 46.22 | 40.29 | 29.8 | 63.98 | 56.35 | 45.3 | |
| SVD-LLMOptimizer=MUON, Compression Rate=40%, Backbone Model=Llama3-1.8B2026.03 | 26.19 | 43.14 | 36.74 | 28.86 | 28.2 | 59.52 | 53.83 | 39.5 | |
| ASVDOptimizer=MUON, Compression Rate=40%, Backbone Model=Llama3-1.8B2026.03 | 24.66 | 31.9 | 29.28 | 7.08 | 27.8 | 55.93 | 50.12 | 32.4 | |
| ASVDOptimizer=ADAMW, Compression Rate=40%, Backbone Model=Llama3-1.8B2026.03 | 22.61 | 37.25 | 30.15 | 6 | 26.4 | 54.52 | 49.88 | 32.4 |