Natural Language Understanding on HULU (val)
89.9HuCOLA AccuracyPULI-LlumiX-Llama-3.1 8B
Evaluation results
| Method | Links | |||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| PULI-LlumiX-Llama-3.1 8BFine-tuning=Best of LoRA and full finetuned2026.01 | 89.9 | 0.692 | 89.7 | 93.6 | 0.872 | 93.6 | 78 | 0.56 | 77 | 89.8 | 0.794 | 89.8 | 38 | -0.282 | 36.7 | 48.5 | 0.274 | 45.9 | 72.9 | 0.485 | 72.1 | |
| Racka-4BFine-tuning=Best of LoRA and full finetuned2026.01 | 86.2 | 0.566 | 85.6 | 79.9 | 0.6 | 79.9 | 76 | 0.514 | 75.1 | 87.9 | 0.755 | 87.9 | 56.7 | 0.103 | 45.5 | 63.9 | 0.474 | 63.7 | 75.1 | 0.502 | 73 | |
| Qwen3-4B-BaseFine-tuning=Best of LoRA and full finetuned2026.01 | 82.5 | 0.404 | 80.3 | 58.5 | 0.171 | 58.4 | 75.4 | 0.508 | 75.1 | 88.7 | 0.772 | 88.7 | 53.7 | -0.06 | 40.7 | 62.9 | 0.473 | 61.1 | 70.2 | 0.378 | 67.3 | |
| Qwen3-4BFine-tuning=Best of LoRA and full finetuned2026.01 | 81.1 | 0.348 | 78.4 | 55.9 | 0.118 | 55.8 | 75.2 | 0.502 | 74.3 | 90.8 | 0.814 | 90.8 | 50.3 | -0.098 | 38.6 | 73.8 | 0.608 | 73.2 | 71.1 | 0.382 | 68.5 |