Language Understanding and Code Generation on Llama 1B Evaluation Suite v3.2
39.33ARCQLoRA w/ TOKENTUNE (Random)
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| QLoRA w/ TOKENTUNE (Random)Backbone=Llama3.2 1B, Ave. Mem.=14.8%, Max. Mem.=14.3%2026.01 | 39.33 | 62.97 | 41.76 | 41.36 | 60.69 | 12.8 | 43.15 | |
| LoHaBackbone=Llama3.2 1B, Ave. Mem.=92.3%, Max. Mem.=99.4%2026.01 | 39.25 | 65.93 | 57.6 | 37.87 | 60.77 | 13.41 | 45.81 | |
| QLoRA w/ TOKENSEEK (Ours)Backbone=Llama3.2 1B, Ave. Mem.=14.8%, Max. Mem.=14.3%2026.01 | 39.08 | 65.98 | 58.03 | 38.65 | 61.33 | 14.63 | 46.28 | |
| QLoRABackbone=Llama3.2 1B, Ave. Mem.=45.6%, Max. Mem.=34.8%2026.01 | 38.82 | 65.26 | 56.39 | 38.85 | 61.33 | 14.02 | 45.78 | |
| LoHa w/ TOKENSEEK (Ours)Backbone=Llama3.2 1B, Ave. Mem.=45.9%, Max. Mem.=28.4%2026.01 | 38.57 | 65.89 | 58.18 | 39.34 | 60.93 | 14.02 | 46.16 | |
| LoHa w/ TOKENTUNE (Random)Backbone=Llama3.2 1B, Ave. Mem.=45.9%, Max. Mem.=28.4%2026.01 | 38.48 | 64.21 | 50.34 | 43.89 | 59.91 | 10.97 | 44.63 |