Zero-shot Task Evaluation on 11 Tasks zero-shot
68.660-shot AverageFine-tuned
Evaluation Results
| Method | Links | |
|---|---|---|
| Fine-tunedModel Backbone=Qwen2-7B, Precision=FP162025.11 | 68.66 | |
| Fine-tunedModel Backbone=LLaMA3.1-8B, Precision=FP162025.11 | 68.03 | |
| Pre-trainedModel Backbone=Qwen2-7B, Precision=FP162025.11 | 67.82 | |
| Pre-trainedModel Backbone=LLaMA3.1-8B, Precision=FP162025.11 | 67.24 | |
| OSTQuantModel Backbone=Qwen2-7B, Precision=W4A42025.11 | 67.01 | |
| CAQModel Backbone=Qwen2-7B, Precision=W4A42025.11 | 66.37 | |
| SpinQuantModel Backbone=Qwen2-7B, Precision=W4A42025.11 | 65.94 | |
| QuaRotModel Backbone=Qwen2-7B, Precision=W4A42025.11 | 65.69 | |
| CAQModel Backbone=LLaMA3.1-8B, Precision=W4A42025.11 | 65.54 | |
| OSTQuantModel Backbone=LLaMA3.1-8B, Precision=W4A42025.11 | 65.32 | |
| SpinQuantModel Backbone=LLaMA3.1-8B, Precision=W4A42025.11 | 64.57 | |
| QuaRotModel Backbone=LLaMA3.1-8B, Precision=W4A42025.11 | 63.02 | |
| Fine-tunedModel Backbone=LLaMA2-7B, Precision=FP162025.11 | 62.32 | |
| DuQuantModel Backbone=LLaMA3.1-8B, Precision=W4A42025.11 | 62.31 | |
| OSTQuantModel Backbone=LLaMA2-7B, Precision=W4A42025.11 | 60.17 | |
| Pre-trainedModel Backbone=LLaMA2-7B, Precision=FP162025.11 | 60.14 | |
| DuQuantModel Backbone=LLaMA2-7B, Precision=W4A42025.11 | 59.2 | |
| CAQModel Backbone=LLaMA2-7B, Precision=W4A42025.11 | 59.05 | |
| SpinQuantModel Backbone=LLaMA2-7B, Precision=W4A42025.11 | 58.61 | |
| QuaRotModel Backbone=LLaMA2-7B, Precision=W4A42025.11 | 58.36 | |
| GPTQModel Backbone=LLaMA3.1-8B, Precision=W4A42025.11 | 36.45 | |
| RTNModel Backbone=LLaMA3.1-8B, Precision=W4A42025.11 | 34.92 | |
| RTNModel Backbone=LLaMA2-7B, Precision=W4A42025.11 | 32.56 | |
| GPTQModel Backbone=LLaMA2-7B, Precision=W4A42025.11 | 32.56 | |
| RTNModel Backbone=Qwen2-7B, Precision=W4A42025.11 | 30.28 | |
| GPTQModel Backbone=Qwen2-7B, Precision=W4A42025.11 | 29.86 |