# Factual Knowledge Evaluation on Wikidata knowledge infusion Source: [Factual Knowledge Evaluation on Wikidata knowledge infusion]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:30:53.957Z Evaluation of model performance on factual knowledge assessment using the Wikidata knowledge infusion dataset, measured by accuracy, hit rate, MRR, and probability scores. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Factual Knowledge Evaluation | | Dataset (including any stated split) | Wikidata knowledge infusion | | Metric | Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 18 of 18 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Accuracy](): Higher is better; unit: PERCENT. - [Hit Rate](): Higher is better; unit: PERCENT. - [MRR](): Higher is better; unit: PERCENT. - [Probability Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | Accuracy | Result date | Report paper | | --- | --- | --- | --- | | PretrainRL | 64.69 | Not recorded | 69818194a8b01e76cc8e2681 | | Iterative RPO | 58.92 | Not recorded | 69818194a8b01e76cc8e2681 | | DPO | 54.42 | Not recorded | 69818194a8b01e76cc8e2681 | | Continual Training (CT) | 53.16 | Not recorded | 69818194a8b01e76cc8e2681 | | PretrainRL | 47.58 | Not recorded | 69818194a8b01e76cc8e2681 | | PretrainRL | 46.2 | Not recorded | 69818194a8b01e76cc8e2681 | | DPO | 38.51 | Not recorded | 69818194a8b01e76cc8e2681 | | Continual Training (CT) | 37.42 | Not recorded | 69818194a8b01e76cc8e2681 | | DPO | 33.84 | Not recorded | 69818194a8b01e76cc8e2681 | | Iterative RPO | 33.5 | Not recorded | 69818194a8b01e76cc8e2681 | | Continual Training (CT) | 31.45 | Not recorded | 69818194a8b01e76cc8e2681 | | Iterative RPO | 30.23 | Not recorded | 69818194a8b01e76cc8e2681 | | Prompt (CoT) | 27.71 | Not recorded | 69818194a8b01e76cc8e2681 | | Llama3-8B | 27.6 | Not recorded | 69818194a8b01e76cc8e2681 | | Qwen3-8B | 23.96 | Not recorded | 69818194a8b01e76cc8e2681 | | Prompt (CoT) | 23.96 | Not recorded | 69818194a8b01e76cc8e2681 | | Qwen3-4B | 20.65 | Not recorded | 69818194a8b01e76cc8e2681 | | Prompt (CoT) | 20.02 | Not recorded | 69818194a8b01e76cc8e2681 | ### PretrainRL Accuracy: 64.69. Result ID: 6996544b5698665ec4995f86. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Llama3-8B","Training Method":"PretrainRL","Decoding Strategy":"Beam Search"}. This record does not assert independent reproduction. ### Iterative RPO Accuracy: 58.92. Result ID: 6996544b5698665ec4995f85. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Llama3-8B","Training Method":"Iterative RPO","Decoding Strategy":"Beam Search"}. This record does not assert independent reproduction. ### DPO Accuracy: 54.42. Result ID: 6996544b5698665ec4995f83. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Llama3-8B","Training Method":"DPO","Decoding Strategy":"Beam Search"}. This record does not assert independent reproduction. ### Continual Training (CT) Accuracy: 53.16. Result ID: 6996544b5698665ec4995f84. [Reporting paper]() Recorded evaluation configuration: {"Base Model":"Llama3-8B","Training Method":"Continual Training","Decoding Strategy":"Beam Search"}. This record does not assert independent reproduction. ### PretrainRL Accuracy: 47.58. Result ID: 6996544b5698665ec4995f80. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen3-8B","Training Method":"PretrainRL","Decoding Strategy":"Beam Search"}. This record does not assert independent reproduction. ### PretrainRL Accuracy: 46.2. Result ID: 6996544b5698665ec4995f7a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen3-4B","Training Method":"PretrainRL","Decoding Strategy":"Beam Search"}. This record does not assert independent reproduction. ### DPO Accuracy: 38.51. Result ID: 6996544b5698665ec4995f7d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen3-8B","Training Method":"DPO","Decoding Strategy":"Beam Search"}. This record does not assert independent reproduction. ### Continual Training (CT) Accuracy: 37.42. Result ID: 6996544b5698665ec4995f78. [Reporting paper]() Recorded evaluation configuration: {"Base Model":"Qwen3-4B","Training Method":"Continual Training","Decoding Strategy":"Beam Search"}. This record does not assert independent reproduction. ### DPO Accuracy: 33.84. Result ID: 6996544b5698665ec4995f77. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen3-4B","Training Method":"DPO","Decoding Strategy":"Beam Search"}. This record does not assert independent reproduction. ### Iterative RPO Accuracy: 33.5. Result ID: 6996544b5698665ec4995f7f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen3-8B","Training Method":"Iterative RPO","Decoding Strategy":"Beam Search"}. This record does not assert independent reproduction. ### Continual Training (CT) Accuracy: 31.45. Result ID: 6996544b5698665ec4995f7e. [Reporting paper]() Recorded evaluation configuration: {"Base Model":"Qwen3-8B","Training Method":"Continual Training","Decoding Strategy":"Beam Search"}. This record does not assert independent reproduction. ### Iterative RPO Accuracy: 30.23. Result ID: 6996544b5698665ec4995f79. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen3-4B","Training Method":"Iterative RPO","Decoding Strategy":"Beam Search"}. This record does not assert independent reproduction. ### Prompt (CoT) Accuracy: 27.71. Result ID: 6996544b5698665ec4995f82. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Llama3-8B","Training Method":"Prompting","Decoding Strategy":"Chain-of-Thought"}. This record does not assert independent reproduction. ### Llama3-8B Accuracy: 27.6. Result ID: 6996544b5698665ec4995f81. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Base Model":"Llama3-8B","Training Method":"Base Model","Decoding Strategy":"Beam Search"}. This record does not assert independent reproduction. ### Qwen3-8B Accuracy: 23.96. Result ID: 6996544b5698665ec4995f7b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen3-8B","Training Method":"Base Model","Decoding Strategy":"Beam Search"}. This record does not assert independent reproduction. ### Prompt (CoT) Accuracy: 23.96. Result ID: 6996544b5698665ec4995f7c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen3-8B","Training Method":"Prompting","Decoding Strategy":"Chain-of-Thought"}. This record does not assert independent reproduction. ### Qwen3-4B Accuracy: 20.65. Result ID: 6996544b5698665ec4995f75. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen3-4B","Training Method":"Base Model","Decoding Strategy":"Beam Search"}. This record does not assert independent reproduction. ### Prompt (CoT) Accuracy: 20.02. Result ID: 6996544b5698665ec4995f76. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen3-4B","Training Method":"Prompting","Decoding Strategy":"Chain-of-Thought"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.