Downstream Task Evaluation on 10 Downstream Tasks
52.79Average AccuracyMeSH
Evaluation results
| Method | Links | ||
|---|---|---|---|
| MeSHModel Size=Pythia-1.4B, Scheme=Recursive, Layers=4+8R2+4, Variant=+mesh, Parameter reduction percentage=-33.3%, Evaluation Protocol=5-shot2025.10 | 52.79 | 0.86 | |
| MeSHModel Size=Pythia-1.4B, Scheme=Recursive, Layers=4+8R2+4, Variant=+mesh, Parameter reduction percentage=-33.3%, Evaluation Protocol=0-shot2025.10 | 50.56 | 1.06 | |
| VanillaModel Size=Pythia-1.4B, Scheme=Vanilla, Layers=24, Evaluation Protocol=0-shot2025.10 | 49.5 | — | |
| MeSHModel Size=Pythia-410M, Scheme=Recursive, Layers=3+6R3+3, Variant=+mesh, Parameter reduction percentage=-50.0%, Evaluation Protocol=5-shot2025.10 | 46.04 | 0.73 | |
| MeSHModel Size=Pythia-410M, Scheme=Recursive, Layers=4+8R2+4, Variant=+mesh, Parameter reduction percentage=-33.3%, Evaluation Protocol=0-shot2025.10 | 44.12 | 0.25 | |
| VanillaModel Size=Pythia-410M, Scheme=Vanilla, Layers=24, Evaluation Protocol=0-shot2025.10 | 43.87 | — | |
| MeSHModel Size=160M, Scheme=Recursive, Layers=2+4R2+2, Variant=+mesh, Parameter reduction percentage=-33.3%, Evaluation Protocol=5-shot2025.10 | 40.6 | 0.06 | |
| VanillaModel Size=160M, Scheme=Vanilla, Layers=12, Evaluation Protocol=0-shot2025.10 | 39.88 | — | |
| MeSHModel Size=160M, Scheme=Recursive, Layers=2+4R2+2, Variant=+mesh, Parameter reduction percentage=-33.3%, Evaluation Protocol=0-shot2025.10 | 39.41 | -0.47 |