Question Answering on LAMBADA
73.2AccuracyFP16
Evaluation results
| Method | Links | |
|---|---|---|
| FP16Model=LLaMA3.1-8B-Instruct, Precision=MXFP42026.04 | 73.2 | |
| QuaRot*Model=LLaMA3.1-8B-Instruct, Precision=MXFP42026.04 | 72.5 | |
| DuQuant++*Model=LLaMA3.1-8B-Instruct, Precision=MXFP42026.04 | 72.3 | |
| FP16Model=LLaMA3-8B-Instruct, Precision=MXFP42026.04 | 71.5 | |
| QuaRot*Model=LLaMA3-8B-Instruct, Precision=MXFP42026.04 | 71.4 | |
| MR-GPTQModel=LLaMA3.1-8B-Instruct, Precision=MXFP42026.04 | 70.4 | |
| DuQuant++*Model=LLaMA3-8B-Instruct, Precision=MXFP42026.04 | 70.3 | |
| DuQuant++Model=LLaMA3.1-8B-Instruct, Precision=MXFP42026.04 | 70.2 | |
| DuQuant++Model=LLaMA3-8B-Instruct, Precision=MXFP42026.04 | 69.7 | |
| FlatQuantModel=LLaMA3.1-8B-Instruct, Precision=MXFP42026.04 | 69.6 | |
| MR-GPTQModel=LLaMA3-8B-Instruct, Precision=MXFP42026.04 | 68.3 | |
| QuaRotModel=LLaMA3-8B-Instruct, Precision=MXFP42026.04 | 67.4 | |
| FlatQuantModel=LLaMA3-8B-Instruct, Precision=MXFP42026.04 | 67.3 | |
| QuaRotModel=LLaMA3.1-8B-Instruct, Precision=MXFP42026.04 | 62 |