# Factual Question Answering on Factual Category Average (test) Source: [Factual Question Answering on Factual Category Average (test)]() Collection index: [llms.txt]() Data updated: 2026-03-09T07:41:21.692Z Evaluation of model performance on the test split of the Factual Category Average dataset for Factual Question Answering, reporting Accuracy, AUROC, Expected Calibration Error (ECE), and Brier score. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Factual Question Answering | | Dataset (including any stated split) | Factual Category Average (test) | | Metric | Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 18 of 18 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Accuracy](): Higher is better; unit: PERCENT. - [AUROC](): Higher is better; unit: PERCENT. - [ECE](): Lower is better; unit: PERCENT. - [Brier Score](): Lower is better; unit: PERCENT. ## Recorded results | Method | Accuracy | Result date | Report paper | | --- | --- | --- | --- | | RLVR+QuestionProb | 31.38 | Not recorded | 69ae53d72d603e6de3e427e3 | | RLVR+Assessor Model | 31.38 | Not recorded | 69ae53d72d603e6de3e427e3 | | RLVR+Probe | 31.38 | Not recorded | 69ae53d72d603e6de3e427e3 | | RLVR | 30.74 | Not recorded | 69ae53d72d603e6de3e427e3 | | Instruct Model | 30.13 | Not recorded | 69ae53d72d603e6de3e427e3 | | CoCA | 28.85 | Not recorded | 69ae53d72d603e6de3e427e3 | | RLVR+QuestionProb | 26.01 | Not recorded | 69ae53d72d603e6de3e427e3 | | RLVR+Assessor Model | 26.01 | Not recorded | 69ae53d72d603e6de3e427e3 | | RLVR+Probe | 26.01 | Not recorded | 69ae53d72d603e6de3e427e3 | | CoCA | 24.74 | Not recorded | 69ae53d72d603e6de3e427e3 | | RLVR | 23.92 | Not recorded | 69ae53d72d603e6de3e427e3 | | Instruct Model | 22.71 | Not recorded | 69ae53d72d603e6de3e427e3 | | RLVR+QuestionProb | 19.79 | Not recorded | 69ae53d72d603e6de3e427e3 | | RLVR+Assessor Model | 19.79 | Not recorded | 69ae53d72d603e6de3e427e3 | | RLVR+Probe | 19.79 | Not recorded | 69ae53d72d603e6de3e427e3 | | CoCA | 18.89 | Not recorded | 69ae53d72d603e6de3e427e3 | | RLVR | 16.77 | Not recorded | 69ae53d72d603e6de3e427e3 | | Instruct Model | 14.81 | Not recorded | 69ae53d72d603e6de3e427e3 | ### RLVR+QuestionProb Accuracy: 31.38. Result ID: 69ae79916dbb7075b9564db7. [Reporting paper]() Recorded evaluation configuration: {"Model Size":"7B","Backbone":"Qwen2.5-Instruct"}. This record does not assert independent reproduction. ### RLVR+Assessor Model Accuracy: 31.38. Result ID: 69ae79916dbb7075b9564dba. [Reporting paper]() Recorded evaluation configuration: {"Model Size":"7B","Backbone":"Qwen2.5-Instruct"}. This record does not assert independent reproduction. ### RLVR+Probe Accuracy: 31.38. Result ID: 69ae79916dbb7075b9564dbd. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"7B","Backbone":"Qwen2.5-Instruct"}. This record does not assert independent reproduction. ### RLVR Accuracy: 30.74. Result ID: 69ae79916dbb7075b9564db4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"7B","Backbone":"Qwen2.5-Instruct"}. This record does not assert independent reproduction. ### Instruct Model Accuracy: 30.13. Result ID: 69ae79916dbb7075b9564db1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"7B","Backbone":"Qwen2.5-Instruct"}. This record does not assert independent reproduction. ### CoCA Accuracy: 28.85. Result ID: 69ae79916dbb7075b9564dc0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"7B","Backbone":"Qwen2.5-Instruct"}. This record does not assert independent reproduction. ### RLVR+QuestionProb Accuracy: 26.01. Result ID: 69ae79916dbb7075b9564db6. [Reporting paper]() Recorded evaluation configuration: {"Model Size":"3B","Backbone":"Qwen2.5-Instruct"}. This record does not assert independent reproduction. ### RLVR+Assessor Model Accuracy: 26.01. Result ID: 69ae79916dbb7075b9564db9. [Reporting paper]() Recorded evaluation configuration: {"Model Size":"3B","Backbone":"Qwen2.5-Instruct"}. This record does not assert independent reproduction. ### RLVR+Probe Accuracy: 26.01. Result ID: 69ae79916dbb7075b9564dbc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"3B","Backbone":"Qwen2.5-Instruct"}. This record does not assert independent reproduction. ### CoCA Accuracy: 24.74. Result ID: 69ae79916dbb7075b9564dbf. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"3B","Backbone":"Qwen2.5-Instruct"}. This record does not assert independent reproduction. ### RLVR Accuracy: 23.92. Result ID: 69ae79916dbb7075b9564db3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"3B","Backbone":"Qwen2.5-Instruct"}. This record does not assert independent reproduction. ### Instruct Model Accuracy: 22.71. Result ID: 69ae79916dbb7075b9564db0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"3B","Backbone":"Qwen2.5-Instruct"}. This record does not assert independent reproduction. ### RLVR+QuestionProb Accuracy: 19.79. Result ID: 69ae79916dbb7075b9564db5. [Reporting paper]() Recorded evaluation configuration: {"Model Size":"1.5B","Backbone":"Qwen2.5-Instruct"}. This record does not assert independent reproduction. ### RLVR+Assessor Model Accuracy: 19.79. Result ID: 69ae79916dbb7075b9564db8. [Reporting paper]() Recorded evaluation configuration: {"Model Size":"1.5B","Backbone":"Qwen2.5-Instruct"}. This record does not assert independent reproduction. ### RLVR+Probe Accuracy: 19.79. Result ID: 69ae79916dbb7075b9564dbb. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"1.5B","Backbone":"Qwen2.5-Instruct"}. This record does not assert independent reproduction. ### CoCA Accuracy: 18.89. Result ID: 69ae79916dbb7075b9564dbe. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"1.5B","Backbone":"Qwen2.5-Instruct"}. This record does not assert independent reproduction. ### RLVR Accuracy: 16.77. Result ID: 69ae79916dbb7075b9564db2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"1.5B","Backbone":"Qwen2.5-Instruct"}. This record does not assert independent reproduction. ### Instruct Model Accuracy: 14.81. Result ID: 69ae79916dbb7075b9564daf. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"1.5B","Backbone":"Qwen2.5-Instruct"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.