Cloze-style Completion on Lambada OpenAI
75.65AccuracyBF16
Evaluation results
| Method | Links | ||
|---|---|---|---|
| BF16Model=Llama3-8B, A-W Quant Type=BF162026.02 | 75.65 | — | |
| BF16Model=Mistral-7B, A-W Quant Type=BF162026.02 | 75.14 | — | |
| NVFP4+PTSModel=Llama3-8B, A-W Quant Type=NVFP4+PTS2026.02 | 74.85 | -0.8 | |
| BF16Model=Qwen2.5-14B, A-W Quant Type=BF162026.02 | 74.31 | — | |
| NVFP4Model=Llama3-8B, A-W Quant Type=NVFP42026.02 | 74.21 | -1.44 | |
| HiF4Model=Mistral-7B, A-W Quant Type=HiF42026.02 | 74.11 | -1.03 | |
| NVFP4+PTSModel=Mistral-7B, A-W Quant Type=NVFP4+PTS2026.02 | 73.96 | -1.18 | |
| HiF4Model=Llama3-8B, A-W Quant Type=HiF42026.02 | 73.76 | -1.89 | |
| NVFP4Model=Llama2-7B, A-W Quant Type=NVFP42026.02 | 73.69 | 0.02 | |
| BF16Model=Llama2-7B, A-W Quant Type=BF162026.02 | 73.67 | — | |
| HiF4+HiGPTQModel=Llama2-7B, A-W Quant Type=HiF4+HiGPTQ2026.02 | 73.66 | -0.01 | |
| HiF4+HiGPTQModel=Mistral-7B, A-W Quant Type=HiF4+HiGPTQ2026.02 | 73.31 | -1.83 | |
| NVFP4+PTSModel=Llama2-7B, A-W Quant Type=NVFP4+PTS2026.02 | 73.16 | -0.51 | |
| HiF4+HiGPTQModel=Qwen2.5-14B, A-W Quant Type=HiF4+HiGPTQ2026.02 | 73.02 | -1.29 | |
| NVFP4Model=Qwen2.5-14B, A-W Quant Type=NVFP42026.02 | 72.95 | -1.36 | |
| NVFP4+PTSModel=Qwen2.5-14B, A-W Quant Type=NVFP4+PTS2026.02 | 72.87 | -1.44 | |
| HiF4+HiGPTQModel=Llama3-8B, A-W Quant Type=HiF4+HiGPTQ2026.02 | 72.85 | -2.8 | |
| HiF4Model=Qwen2.5-14B, A-W Quant Type=HiF42026.02 | 72.74 | -1.57 | |
| HiF4Model=Llama2-7B, A-W Quant Type=HiF42026.02 | 72.65 | -1.02 | |
| NVFP4Model=Mistral-7B, A-W Quant Type=NVFP42026.02 | 0 | -75.14 |