Few-shot Language Evaluation on ARC, HellaSwag, MMLU, TruthfulQA, and WinoGrande
56.06ARC AccuracyQLoRA
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| QLoRABackbone=Llama2 (7B), Ave. Mem.=53.2%, Max. Mem.=46.5%2026.01 | 56.06 | 78.6 | 65.08 | 43.64 | 69.38 | 62.55 | |
| TOKENSEEKBackbone=Llama2 (7B), Tuning Mode=QLoRA, Ave. Mem.=18.2%, Max. Mem.=12.4%2026.01 | 53.5 | 78.82 | 65.26 | 44.62 | 68.51 | 62.14 | |
| TOKENTUNEBackbone=Llama2 (7B), Tuning Mode=QLoRA, Ave. Mem.=18.2%, Max. Mem.=12.4%2026.01 | 53.16 | 78.76 | 63.64 | 39.58 | 69.22 | 60.87 | |
| Full Parameter TuningBackbone=Llama2 (7B), Ave. Mem.=100%, Max. Mem.=100%2026.01 | 52.39 | 78.97 | 64.44 | 38.97 | 68.9 | 60.73 | |
| TOKENSEEKBackbone=Llama2 (7B), Tuning Mode=Full Parameter, Ave. Mem.=77.9%, Max. Mem.=45.6%2026.01 | 52.22 | 78.96 | 65.28 | 39.95 | 68.43 | 60.97 | |
| TOKENTUNEBackbone=Llama2 (7B), Tuning Mode=Full Parameter, Ave. Mem.=77.9%, Max. Mem.=45.6%2026.01 | 51.71 | 78.35 | 61.56 | 41.88 | 70.01 | 60.7 |