Code Generation on HumanEval (Accuracy (%), Avg.)
58.54Accuracy (%)FP
Evaluation results
| Method | Links | ||
|---|---|---|---|
| FPModel=Dream-Base, Quantization=Full Precision2026.06 | 58.54 | 70.01 | |
| FPModel=Dream-Instruct, Quantization=Full Precision2026.06 | 57.93 | 71.01 | |
| FAIR-CalibModel=Dream-Instruct, Quantization=W4A42026.06 | 44.51 | 66.66 | |
| FAIR-CalibModel=Dream-Base, Quantization=W4A42026.06 | 41.46 | 64.64 | |
| FlatQuantModel=Dream-Instruct, Quantization=W4A42026.06 | 41.46 | 63.98 | |
| FlatQuantModel=Dream-Base, Quantization=W4A42026.06 | 38.41 | 62.08 | |
| QuaRotModel=Dream-Instruct, Quantization=W4A42026.06 | 32.32 | 61.14 | |
| QuaRotModel=Dream-Base, Quantization=W4A42026.06 | 23.17 | 57.49 | |
| RTNModel=Dream-Instruct, Quantization=W4A42026.06 | 13.05 | 45.76 | |
| RTNModel=Dream-Base, Quantization=W4A42026.06 | 6.71 | 42 |