Natural Language Understanding on GLUE8 (val)
0.578CoLA ScoreLORA+
Evaluation Results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| LORA+SETTING=NON-DP, Backbone=Gemma-3-4B-pt2026.05 | 0.578 | 0.95 | 0.84 | 0.862 | 0.807 | 0.845 | 0.851 | 0.838 | 0.769 | |
| RITESETTING=NON-DP, Backbone=Gemma-3-4B-pt2026.05 | 0.515 | 0.947 | 0.883 | 0.873 | 0.821 | 0.846 | 0.889 | 0.895 | 0.782 | |
| ADAMWSETTING=NON-DP, Backbone=Gemma-3-4B-pt2026.05 | 0.504 | 0.954 | 0.831 | 0.863 | 0.766 | 0.813 | 0.846 | 0.848 | 0.752 | |
| LAMBSETTING=NON-DP, Backbone=Gemma-3-4B-pt2026.05 | 0.468 | 0.939 | 0.86 | 0.872 | 0.776 | 0.842 | 0.868 | 0.856 | 0.756 | |
| FFASETTING=NON-DP, Backbone=Gemma-3-4B-pt2026.05 | 0.456 | 0.935 | 0.759 | 0.821 | 0.713 | 0.736 | 0.809 | 0.809 | 0.712 | |
| PRISMSETTING=ε = 6, Backbone=Gemma-3-4B-pt, δ=10^-52026.05 | 0.444 | 0.919 | 0.798 | 0.718 | 0.77 | 0.707 | 0.776 | 0.791 | 0.69 | |
| LORA+SETTING=ε = 6, Backbone=Gemma-3-4B-pt, δ=10^-52026.05 | 0.436 | 0.897 | 0.787 | 0.691 | 0.739 | 0.721 | 0.747 | 0.823 | 0.674 | |
| LORA+SETTING=ε = 3, Backbone=Gemma-3-4B-pt, δ=10^-52026.05 | 0.434 | 0.906 | 0.798 | 0.668 | 0.73 | 0.708 | 0.74 | 0.812 | 0.665 | |
| LAMBSETTING=ε = 6, Backbone=Gemma-3-4B-pt, δ=10^-52026.05 | 0.414 | 0.92 | 0.756 | 0.544 | 0.521 | 0.602 | 0.709 | 0.776 | 0.621 | |
| ADAMWSETTING=ε = 3, Backbone=Gemma-3-4B-pt, δ=10^-52026.05 | 0.41 | 0.903 | 0.778 | 0.622 | 0.555 | 0.633 | 0.718 | 0.812 | 0.634 | |
| ADAMWSETTING=ε = 6, Backbone=Gemma-3-4B-pt, δ=10^-52026.05 | 0.407 | 0.915 | 0.77 | 0.659 | 0.493 | 0.651 | 0.716 | 0.798 | 0.641 | |
| PRISMSETTING=ε = 3, Backbone=Gemma-3-4B-pt, δ=10^-52026.05 | 0.406 | 0.884 | 0.784 | 0.729 | 0.77 | 0.732 | 0.791 | 0.78 | 0.68 | |
| LAMBSETTING=ε = 3, Backbone=Gemma-3-4B-pt, δ=10^-52026.05 | 0.396 | 0.909 | 0.759 | 0.517 | 0.486 | 0.586 | 0.708 | 0.783 | 0.609 | |
| PRISMSETTING=NON-DP, Backbone=Gemma-3-4B-pt2026.05 | 0.392 | 0.921 | 0.857 | 0.822 | 0.797 | 0.814 | 0.834 | 0.798 | 0.737 | |
| FFASETTING=ε = 6, Backbone=Gemma-3-4B-pt, δ=10^-52026.05 | 0.355 | 0.907 | 0.738 | 0.465 | 0.479 | 0.579 | 0.684 | 0.755 | 0.589 | |
| FFASETTING=ε = 3, Backbone=Gemma-3-4B-pt, δ=10^-52026.05 | 0.337 | 0.89 | 0.73 | 0.406 | 0.466 | 0.561 | 0.662 | 0.74 | 0.569 | |
| RITESETTING=ε = 6, Backbone=Gemma-3-4B-pt, δ=10^-52026.05 | 0.235 | 0.787 | 0.635 | 0.268 | 0.5 | 0.482 | 0.562 | 0.657 | 0.49 | |
| RITESETTING=ε = 3, Backbone=Gemma-3-4B-pt, δ=10^-52026.05 | 0.221 | 0.713 | 0.636 | 0.26 | 0.485 | 0.463 | 0.548 | 0.606 | 0.462 |