Common-sense QA on OpenbookQA
52.8AccuracyGemma 7B
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Gemma 7BParameters=7B2024.07 | 52.8 | — | |
| Mixtral 8×22BParameters=8×22B2024.07 | 50.8 | — | |
| OPT-2.7B FP16Model=OPT-2.7B, Precision=FP162023.08 | 49.6 | 26.16 | |
| Llama 3 405BParameters=405B2024.07 | 49.2 | — | |
| Mistral 7BParameters=7B2024.07 | 47.8 | — | |
| Llama 3 70BParameters=70B2024.07 | 47.6 | — | |
| TSLDModel=OPT-2.7B, Quantization=QAT-KD (tensor-wise)2023.08 | 46.81 | 28.93 | |
| GT+LogitModel=OPT-2.7B, Quantization=QAT-KD (tensor-wise)2023.08 | 46.2 | 31.08 | |
| LogitModel=OPT-2.7B, Quantization=QAT-KD (tensor-wise)2023.08 | 45.4 | 29.41 | |
| Llama 3 8BParameters=8B2024.07 | 45 | — |