Factual Knowledge Evaluation on PopQA
35.3AccuracyCorVer
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| CorVerModel=Llama-3.1-8B2026.05 | 35.3 | — | — | — | |
| FoRAGModel=Llama-3.1-8B2026.05 | 32.17 | — | — | — | |
| RLFHModel=Llama-3.1-8B2026.05 | 32.16 | — | — | — | |
| RawModel=Llama-3.1-8B2026.05 | 28.85 | — | — | — | |
| KnowRLModel=Llama-3.1-8B2026.05 | 25.45 | — | — | — | |
| CorVerModel=Llama-3.2-3B2026.05 | 23.75 | — | — | — | |
| FSPOModel=Llama-3.1-8B2026.05 | 23.54 | — | — | — | |
| FoRAGModel=Llama-3.2-3B2026.05 | 22.53 | — | — | — | |
| RLFHModel=Llama-3.2-3B2026.05 | 22.42 | — | — | — | |
| FSPOModel=Qwen3-8B2026.05 | 22.09 | — | — | — | |
| RLFHModel=Qwen3-8B2026.05 | 22.03 | — | — | — | |
| CorVerModel=Qwen3-8B2026.05 | 21.83 | — | — | — | |
| FoRAGModel=Qwen3-8B2026.05 | 21.8 | — | — | — | |
| RawModel=Qwen3-8B2026.05 | 20.34 | — | — | — | |
| CorVerModel=Qwen3-4B2026.05 | 19.33 | — | — | — | |
| FSPOModel=Qwen3-4B2026.05 | 18.76 | — | — | — | |
| FoRAGModel=Qwen3-4B2026.05 | 18.49 | — | — | — | |
| RLFHModel=Qwen3-4B2026.05 | 18.4 | — | — | — | |
| Llama 2Parameters=7B, Shots=102026.01 | 18 | — | — | — | |
| RawModel=Qwen3-4B2026.05 | 17.51 | — | — | — | |
| KnowRLModel=Qwen3-8B2026.05 | 16.97 | — | — | — | |
| OLMo 2Parameters=7B, Shots=102026.01 | 16 | — | — | — | |
| RawModel=Llama-3.2-3B2026.05 | 15.92 | — | — | — | |
| KnowRLModel=Llama-3.2-3B2026.05 | 15.28 | — | — | — | |
| UnfilteredShots=102026.01 | 11 | — | — | — | |
| Alignment UpsampledStrategy=Mid, Shots=102026.01 | 11 | — | — | — | |
| Filtered + Alignment UpsampledStrategy=E2E, Shots=102026.01 | 11 | — | — | — | |
| OLMo 3Parameters=7B, Shots=102026.01 | 11 | — | — | — | |
| KnowRLModel=Qwen3-4B2026.05 | 10.2 | — | — | — | |
| FilteredShots=102026.01 | 10 | — | — | — | |
| Alignment UpsampledStrategy=E2E, Shots=102026.01 | 10 | — | — | — | |
| Alignment UpsampledStrategy=CPT, Shots=102026.01 | 10 | — | — | — | |
| Misalignment UpsampledStrategy=Mid, Shots=102026.01 | 10 | — | — | — | |
| Misalignment UpsampledStrategy=CPT, Shots=102026.01 | 10 | — | — | — | |
| Filtered + Alignment UpsampledStrategy=Mid, Shots=102026.01 | 9 | — | — | — | |
| Filtered + Alignment UpsampledStrategy=CPT, Shots=102026.01 | 9 | — | — | — | |
| Misalignment UpsampledStrategy=E2E, Shots=102026.01 | 9 | — | — | — | |
| FSPOModel=Llama-3.2-3B2026.05 | 4.44 | — | — | — | |
| PretrainRLBase Model=Llama3-8B, Training Method=PretrainRL, Decoding Strategy=Beam Search2026.02 | 0.5016 | 0.5138 | 0.3863 | 0.3067 | |
| PretrainRLBase Model=Qwen3-8B, Training Method=PretrainRL, Decoding Strategy=Beam Search2026.02 | 0.4861 | 0.7492 | 0.5151 | 0.4048 | |
| Iterative RPOBase Model=Qwen3-8B, Training Method=Iterative RPO, Decoding Strategy=Beam Search2026.02 | 0.4123 | 0.4813 | 0.3228 | 0.4083 | |
| Continual Training (CT)Base Model=Qwen3-8B, Training Method=Continual Training, Decoding Strategy=Beam Search2026.02 | 0.3853 | 0.6917 | 0.4193 | 0.275 | |
| PretrainRLBase Model=Qwen3-4B, Training Method=PretrainRL, Decoding Strategy=Beam Search2026.02 | 0.3769 | 0.6639 | 0.4051 | 0.3142 | |
| Iterative RPOBase Model=Llama3-8B, Training Method=Iterative RPO, Decoding Strategy=Beam Search2026.02 | 0.3383 | 0.3263 | 0.2264 | 0.2817 | |
| Iterative RPOBase Model=Qwen3-4B, Training Method=Iterative RPO, Decoding Strategy=Beam Search2026.02 | 0.325 | 0.493 | 0.2512 | 0.2077 | |
| Continual Training (CT)Base Model=Qwen3-4B, Training Method=Continual Training, Decoding Strategy=Beam Search2026.02 | 0.2997 | 0.6036 | 0.3294 | 0.2092 | |
| Prompt (CoT)Base Model=Llama3-8B, Training Method=Prompting, Decoding Strategy=Chain-of-Thought2026.02 | 0.2428 | 0.4968 | 0.2706 | 0.2132 | |
| Llama3-8BBase Model=Llama3-8B, Training Method=Base Model, Decoding Strategy=Beam Search2026.02 | 0.2366 | 0.4869 | 0.2635 | 0.231 | |
| Qwen3-8BBase Model=Qwen3-8B, Training Method=Base Model, Decoding Strategy=Beam Search2026.02 | 0.1632 | 0.3706 | 0.186 | 0.186 | |
| DPOBase Model=Llama3-8B, Training Method=DPO, Decoding Strategy=Beam Search2026.02 | 0.1568 | 0.2602 | 0.1693 | 0.4228 | |
| Prompt (CoT)Base Model=Qwen3-8B, Training Method=Prompting, Decoding Strategy=Chain-of-Thought2026.02 | 0.1554 | 0.3757 | 0.1797 | 0.1214 | |
| Continual Training (CT)Base Model=Llama3-8B, Training Method=Continual Training, Decoding Strategy=Beam Search2026.02 | 0.134 | 0.3318 | 0.1653 | 0.201 | |
| DPOBase Model=Qwen3-8B, Training Method=DPO, Decoding Strategy=Beam Search2026.02 | 0.1308 | 0.2217 | 0.1392 | 0.4211 | |
| Qwen3-4BBase Model=Qwen3-4B, Training Method=Base Model, Decoding Strategy=Beam Search2026.02 | 0.1305 | 0.316 | 0.1517 | 0.1238 | |
| Prompt (CoT)Base Model=Qwen3-4B, Training Method=Prompting, Decoding Strategy=Chain-of-Thought2026.02 | 0.1092 | 0.3247 | 0.1478 | 0.1092 | |
| DPOBase Model=Qwen3-4B, Training Method=DPO, Decoding Strategy=Beam Search2026.02 | 0.0967 | 0.1806 | 0.1102 | 0.3388 |