Long-form Factuality on Factory
30.7R@64KLCF-zero
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| KLCF-zeroBackbone=Qwen2.5-32B, Setting=zero-shot2025.09 | 30.7 | 43.4 | 32.2 | 71.3 | |
| KLCF-zeroModel Backbone=Qwen2.5-14B-Base, Training Stage=Base Model2025.09 | 29.6 | 35 | 30.9 | 65.5 | |
| KLCF-zeroBackbone=Qwen2.5-7B, Setting=zero-shot2025.09 | 25.8 | 30.1 | 27.1 | 78.9 | |
| GRPO&FActScoreModel Backbone=Qwen2.5-14B-Base, Training Stage=Base Model2025.09 | 25.3 | 31.8 | 27.8 | 35.9 | |
| Base (10-shot)Model Backbone=Qwen2.5-14B-Base, Training Stage=Base Model2025.09 | 23.9 | 30.2 | 26.2 | — | |
| BaseBackbone=Qwen2.5-32B, Setting=10-shot prompting2025.09 | 23.4 | 31.8 | 25.6 | — | |
| KLCFModel Backbone=DeepSeek-R1-Distill-Qwen-14B, Training Stage=SFT Model2025.09 | 22.1 | 38.3 | 24.2 | 66.5 | |
| GRPO&FActScore&No.ClaimsModel Backbone=DeepSeek-R1-Distill-Qwen-14B, Training Stage=SFT Model2025.09 | 21.3 | 36.1 | 24.7 | 61.5 | |
| BaseBackbone=Qwen2.5-7B, Setting=10-shot prompting2025.09 | 20.3 | 26.7 | 22.3 | — | |
| CoVeBackbone=Qwen2.5-32B2025.09 | 19.1 | 34.6 | 22.9 | 15.3 | |
| CoVeModel Backbone=Qwen2.5-14B-Base, Training Stage=Base Model2025.09 | 17.9 | 30.4 | 20.6 | 17.9 | |
| CoVeBackbone=Qwen2.5-7B2025.09 | 17.1 | 27.8 | 19.2 | 11.8 | |
| DPO&FActScoreModel Backbone=DeepSeek-R1-Distill-Qwen-14B, Training Stage=SFT Model2025.09 | 15.8 | 30.8 | 19 | 44.4 | |
| DPO&KLCModel Backbone=DeepSeek-R1-Distill-Qwen-14B, Training Stage=SFT Model2025.09 | 15.7 | 31.2 | 18.6 | 52.4 | |
| IntuitorModel Backbone=DeepSeek-R1-Distill-Qwen-14B, Training Stage=SFT Model2025.09 | 14.5 | 29.4 | 18.3 | 41.8 | |
| GRPO&FActScoreModel Backbone=DeepSeek-R1-Distill-Qwen-14B, Training Stage=SFT Model2025.09 | 14.3 | 35.1 | 18.4 | 30.2 | |
| SFT (Distill)Model Backbone=DeepSeek-R1-Distill-Qwen-14B, Training Stage=SFT Model2025.09 | 13.8 | 28.6 | 17.5 | — | |
| Self-Eval-P(True)Model Backbone=Qwen2.5-14B-Base, Training Stage=Base Model2025.09 | 12.6 | 33.7 | 17.3 | 25.9 |