# Fact Verification on FEVER (Oracle Gap) Source: [Fact Verification on FEVER (Oracle Gap)]() Collection index: [llms.txt]() Data updated: 2026-06-25T08:00:20.895Z Evaluation of model performance on the FEVER dataset for Fact Verification, specifically reporting the Oracle Gap metric. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Fact Verification | | Dataset (including any stated split) | FEVER | | Metric | Oracle Gap (pp) | | Unit | SCALAR | | Direction | Lower is better | | Results loaded | 12 of 12 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Oracle Gap (pp)](): Lower is better; unit: SCALAR. ## Recorded results | Method | Oracle Gap (pp) | Result date | Report paper | | --- | --- | --- | --- | | Llama-3.1-8B | 0 | Not recorded | 6a3cb63c206542251d7ca6a8 | | Mistral-7B-v0.3 | 0 | Not recorded | 6a3cb63c206542251d7ca6a8 | | Qwen3-8B | 0 | Not recorded | 6a3cb63c206542251d7ca6a8 | | Gemma-2-9B | 0 | Not recorded | 6a3cb63c206542251d7ca6a8 | | Mean Gap: FEVER | 0.46 | Not recorded | 6a3cb63c206542251d7ca6a8 | | Gemma-2-9B | 0.5 | Not recorded | 6a3cb63c206542251d7ca6a8 | | Mistral-7B-v0.3 | 0.7 | Not recorded | 6a3cb63c206542251d7ca6a8 | | Qwen3-8B | 1.1 | Not recorded | 6a3cb63c206542251d7ca6a8 | | Mean Gap: FEVER | 1.4 | Not recorded | 6a3cb63c206542251d7ca6a8 | | Qwen2.5-7B | 1.6 | Not recorded | 6a3cb63c206542251d7ca6a8 | | Qwen2.5-7B | 2.3 | Not recorded | 6a3cb63c206542251d7ca6a8 | | Llama-3.1-8B | 3.1 | Not recorded | 6a3cb63c206542251d7ca6a8 | ### Llama-3.1-8B Oracle Gap (pp): 0. Result ID: 6a3ce002fbc373592c51e47c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Routing Strategy":"RSC-Based"}. This record does not assert independent reproduction. ### Mistral-7B-v0.3 Oracle Gap (pp): 0. Result ID: 6a3ce002fbc373592c51e47e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Routing Strategy":"RSC-Based"}. This record does not assert independent reproduction. ### Qwen3-8B Oracle Gap (pp): 0. Result ID: 6a3ce002fbc373592c51e480. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Routing Strategy":"RSC-Based"}. This record does not assert independent reproduction. ### Gemma-2-9B Oracle Gap (pp): 0. Result ID: 6a3ce002fbc373592c51e484. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Routing Strategy":"RSC-Based"}. This record does not assert independent reproduction. ### Mean Gap: FEVER Oracle Gap (pp): 0.46. Result ID: 6a3ce002fbc373592c51e486. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Routing Strategy":"RSC-Based"}. This record does not assert independent reproduction. ### Gemma-2-9B Oracle Gap (pp): 0.5. Result ID: 6a3ce002fbc373592c51e483. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Routing Strategy":"NF-Only"}. This record does not assert independent reproduction. ### Mistral-7B-v0.3 Oracle Gap (pp): 0.7. Result ID: 6a3ce002fbc373592c51e47d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Routing Strategy":"NF-Only"}. This record does not assert independent reproduction. ### Qwen3-8B Oracle Gap (pp): 1.1. Result ID: 6a3ce002fbc373592c51e47f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Routing Strategy":"NF-Only"}. This record does not assert independent reproduction. ### Mean Gap: FEVER Oracle Gap (pp): 1.4. Result ID: 6a3ce002fbc373592c51e485. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Routing Strategy":"NF-Only"}. This record does not assert independent reproduction. ### Qwen2.5-7B Oracle Gap (pp): 1.6. Result ID: 6a3ce002fbc373592c51e481. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Routing Strategy":"NF-Only"}. This record does not assert independent reproduction. ### Qwen2.5-7B Oracle Gap (pp): 2.3. Result ID: 6a3ce002fbc373592c51e482. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Routing Strategy":"RSC-Based"}. This record does not assert independent reproduction. ### Llama-3.1-8B Oracle Gap (pp): 3.1. Result ID: 6a3ce002fbc373592c51e47b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Routing Strategy":"NF-Only"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.