Question Answering on PIQA (Accuracy and Normalized Accuracy)
73.61Normalized AccuracySoftmax
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SoftmaxModel Scale=1.8B2025.04 | 73.61 | 73.78 | |
| SoftmaxQuantization=2-bit2025.04 | 73.45 | 73.78 | |
| SoftmaxQuantization=3-bit2025.04 | 73.45 | 73.78 | |
| SoftpickModel Scale=1.8B2025.04 | 70.89 | 71.27 | |
| SoftpickQuantization=2-bit2025.04 | 70.89 | 71.38 | |
| SoftpickQuantization=3-bit2025.04 | 70.89 | 71.38 | |
| SoftmaxModel Scale=340M2025.04 | 66.59 | 66.97 | |
| SoftpickModel Scale=340M2025.04 | 66.49 | 66.59 |