Factuality Evaluation on Factuality Benchmarks Aggregate including FActScore, HaluEval (test)
0.84Generation Quality (Std. Prefix)Self-Improving Pretraining
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Self-Improving PretrainingBackbone=Llama 1.4B, Training Dataset=SlimPajama, Pretraining Objective=Factuality2026.01 | 0.84 | 0.505 | 0.576 | |
| Llama BaseBackbone=Llama 1.4B, Pretraining Objective=Base2026.01 | 0.5 | 0.476 | 0.423 | |
| Llama Pretrain BaselineBackbone=Llama 1.4B, Training Dataset=SlimPajama, Pretraining Objective=Next Token Prediction2026.01 | 0.49 | 0.468 | 0.44 |