Natural Language Understanding on GLUE 2018 (test)
88.5QNLI AccuracyFlexLoRA
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| FlexLoRABackbone=RoBERTa (125M), GPU hours=12.6h2025.01 | 88.5 | 81.2 | 77.5 | 63 | 62.2 | 92.8 | 87.4 | 78.9 | |
| FSLoRABackbone=RoBERTa (125M), GPU hours=10.9h2025.01 | 88 | 87.3 | 82.2 | 76.4 | 69.8 | 93.5 | 85.8 | 83.3 | |
| HeteroLoRABackbone=RoBERTa (125M), GPU hours=10.7h2025.01 | 87.5 | 84.4 | 75.3 | 66.3 | 69 | 89.5 | 85.3 | 79.6 | |
| FLoRABackbone=RoBERTa (125M), GPU hours=12.3h2025.01 | 87.2 | 78.1 | 77.4 | 74.6 | 54.4 | 93.4 | 87.1 | 78.9 | |
| RAVANBackbone=RoBERTa (125M), GPU hours=12.3h2025.01 | 83.4 | 76.2 | 71.8 | 66.3 | 63.1 | 92.1 | 83.4 | 76.6 | |
| Base modelBackbone=RoBERTa (125M)2025.01 | 49.5 | 52 | 30.9 | 32.6 | 47.3 | 50.9 | 63.2 | 46.6 |