Question Answering on Commonsense QA (Comprehensive Suite)
80.85PIQAFP16
Evaluation Results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| FP16Backbone=LLaMA3-8B, Precision=FP162024.06 | 80.85 | 77.78 | 53.41 | 81.28 | 79.16 | 72.84 | 74.22 | |
| DuQuantBackbone=LLaMA3-8B, Weight-Activation Bits=W6A62024.06 | 80.2 | 77.27 | 52.05 | 80.12 | 79.14 | 72.77 | 73.59 | |
| DuQuant+LWCBackbone=LLaMA3-8B, Weight-Activation Bits=W6A62024.06 | 79.71 | 77.57 | 53.07 | 80 | 78.7 | 73.09 | 73.69 | |
| SmoothQuantBackbone=LLaMA3-8B, Weight-Activation Bits=W6A62024.06 | 78.94 | 75.88 | 49.49 | 77.58 | 77.39 | 70.8 | 71.68 | |
| OmniQuantBackbone=LLaMA3-8B, Weight-Activation Bits=W6A62024.06 | 78.9 | 73.95 | 47.35 | 74.95 | 76.77 | 70.56 | 70.41 | |
| AffineQuantBackbone=LLaMA3-8B, Weight-Activation Bits=W6A62024.06 | 78.73 | 73.32 | 46.08 | 74.59 | 77.08 | 70.88 | 70.11 | |
| DuQuant+LWCBackbone=LLaMA3-8B, Weight-Activation Bits=W4A42024.06 | 76.22 | 70.41 | 43.69 | 74.34 | 73.87 | 67.8 | 67.72 | |
| DuQuantBackbone=LLaMA3-8B, Weight-Activation Bits=W4A42024.06 | 75.68 | 68.48 | 41.81 | 71.99 | 73.07 | 66.22 | 66.21 | |
| AtomBackbone=LLaMA3-8B, Weight-Activation Bits=W4A42024.06 | 62.95 | 49.45 | 30.12 | 60.31 | 53.75 | 56.04 | 52.1 | |
| SmoothQuantBackbone=LLaMA3-8B, Weight-Activation Bits=W4A42024.06 | 54.57 | 31.9 | 24.23 | 52.72 | 31.26 | 51.14 | 40.97 | |
| AffineQuantBackbone=LLaMA3-8B, Weight-Activation Bits=W4A42024.06 | 50.71 | 25.93 | 26.02 | 40.55 | 26.07 | 48.46 | 36.29 | |
| OmniQuantBackbone=LLaMA3-8B, Weight-Activation Bits=W4A42024.06 | 50.22 | 26.94 | 24.57 | 37.98 | 26.55 | 50.2 | 36.08 |