# Reading Comprehension on QuAC Source: [Reading Comprehension on QuAC]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:43:17.044Z Evaluation of reading comprehension models on the QuAC dataset, typically measured using F1 score. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Reading Comprehension | | Dataset (including any stated split) | QuAC | | Metric | F1 Score | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 28 of 28 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [F1 Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | F1 Score | Result date | Report paper | | --- | --- | --- | --- | | Fine-tuned SOTA | 74.4 | Not recorded | 68fa680c1e209131cea19d53 | | Llama 2 | 49.3 | Not recorded | 68fa701f1e209131cea492fe | | Llama 2 | 44.8 | Not recorded | 68fa701f1e209131cea492fe | | Llama 2 | 44.4 | Not recorded | 68fa701f1e209131cea492fe | | GPT-3 | 44.3 | Not recorded | 68fa680c1e209131cea19d53 | | Llama 1 | 44.1 | Not recorded | 68fa701f1e209131cea492fe | | GPT-3 | 43.3 | Not recorded | 68fa680c1e209131cea19d53 | | Falcon | 43.3 | Not recorded | 68fa701f1e209131cea492fe | | Llama 2 | 42.9 | Not recorded | 68fa701f1e209131cea492fe | | Llama 2 | 42.7 | Not recorded | 68fa701f1e209131cea492fe | | Llama 2 | 42.4 | Not recorded | 68fa701f1e209131cea492fe | | GPT-3 | 41.5 | Not recorded | 68fa680c1e209131cea19d53 | | Falcon | 41.2 | Not recorded | 68fa701f1e209131cea492fe | | MPT | 41.1 | Not recorded | 68fa701f1e209131cea492fe | | Llama 1 | 41 | Not recorded | 68fa701f1e209131cea492fe | | MPT | 40.4 | Not recorded | 68fa701f1e209131cea492fe | | Llama 1 | 40.3 | Not recorded | 68fa701f1e209131cea492fe | | Llama 1 | 39.9 | Not recorded | 68fa701f1e209131cea492fe | | Llama 1 | 39.8 | Not recorded | 68fa701f1e209131cea492fe | | Llama 2 | 39.7 | Not recorded | 68fa701f1e209131cea492fe | | Llama 2 | 39.4 | Not recorded | 68fa701f1e209131cea492fe | | Llama 1 | 38.9 | Not recorded | 68fa701f1e209131cea492fe | | MPT | 38 | Not recorded | 68fa701f1e209131cea492fe | | MPT | 37.7 | Not recorded | 68fa701f1e209131cea492fe | | Llama 1 | 36.5 | Not recorded | 68fa701f1e209131cea492fe | | Llama 1 | 32 | Not recorded | 68fa701f1e209131cea492fe | | Falcon | 24 | Not recorded | 68fa701f1e209131cea492fe | | Falcon | 18.8 | Not recorded | 68fa701f1e209131cea492fe | ### Fine-tuned SOTA F1 Score: 74.4. Result ID: 698d57ce8b517eb43320c588. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Setting":"Fine-tuned"}. This record does not assert independent reproduction. ### Llama 2 F1 Score: 49.3. Result ID: 69911f7f3a6c31484905db5c. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"70B","shot":"1-shot"}. This record does not assert independent reproduction. ### Llama 2 F1 Score: 44.8. Result ID: 69911f7f3a6c31484905db58. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"13B","shot":"1-shot"}. This record does not assert independent reproduction. ### Llama 2 F1 Score: 44.4. Result ID: 69911f7f3a6c31484905db5a. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"34B","shot":"1-shot"}. This record does not assert independent reproduction. ### GPT-3 F1 Score: 44.3. Result ID: 698d57ce8b517eb43320c58b. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Setting":"Few-Shot"}. This record does not assert independent reproduction. ### Llama 1 F1 Score: 44.1. Result ID: 69911f7f3a6c31484905db51. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"33B","shot":"0-shot"}. This record does not assert independent reproduction. ### GPT-3 F1 Score: 43.3. Result ID: 698d57ce8b517eb43320c58a. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Setting":"One-Shot"}. This record does not assert independent reproduction. ### Falcon F1 Score: 43.3. Result ID: 69911f7f3a6c31484905db4c. [Reporting paper]() Recorded evaluation configuration: {"Size":"40B","shot":"1-shot"}. This record does not assert independent reproduction. ### Llama 2 F1 Score: 42.9. Result ID: 69911f7f3a6c31484905db59. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"34B","shot":"0-shot"}. This record does not assert independent reproduction. ### Llama 2 F1 Score: 42.7. Result ID: 69911f7f3a6c31484905db57. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"13B","shot":"0-shot"}. This record does not assert independent reproduction. ### Llama 2 F1 Score: 42.4. Result ID: 69911f7f3a6c31484905db5b. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"70B","shot":"0-shot"}. This record does not assert independent reproduction. ### GPT-3 F1 Score: 41.5. Result ID: 698d57ce8b517eb43320c589. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Setting":"Zero-Shot"}. This record does not assert independent reproduction. ### Falcon F1 Score: 41.2. Result ID: 69911f7f3a6c31484905db4b. [Reporting paper]() Recorded evaluation configuration: {"Size":"40B","shot":"0-shot"}. This record does not assert independent reproduction. ### MPT F1 Score: 41.1. Result ID: 69911f7f3a6c31484905db48. [Reporting paper]() Recorded evaluation configuration: {"Size":"30B","shot":"1-shot"}. This record does not assert independent reproduction. ### Llama 1 F1 Score: 41. Result ID: 69911f7f3a6c31484905db53. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"65B","shot":"0-shot"}. This record does not assert independent reproduction. ### MPT F1 Score: 40.4. Result ID: 69911f7f3a6c31484905db47. [Reporting paper]() Recorded evaluation configuration: {"Size":"30B","shot":"0-shot"}. This record does not assert independent reproduction. ### Llama 1 F1 Score: 40.3. Result ID: 69911f7f3a6c31484905db52. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"33B","shot":"1-shot"}. This record does not assert independent reproduction. ### Llama 1 F1 Score: 39.9. Result ID: 69911f7f3a6c31484905db4f. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"13B","shot":"0-shot"}. This record does not assert independent reproduction. ### Llama 1 F1 Score: 39.8. Result ID: 69911f7f3a6c31484905db54. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"65B","shot":"1-shot"}. This record does not assert independent reproduction. ### Llama 2 F1 Score: 39.7. Result ID: 69911f7f3a6c31484905db56. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"7B","shot":"1-shot"}. This record does not assert independent reproduction. ### Llama 2 F1 Score: 39.4. Result ID: 69911f7f3a6c31484905db55. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"7B","shot":"0-shot"}. This record does not assert independent reproduction. ### Llama 1 F1 Score: 38.9. Result ID: 69911f7f3a6c31484905db4d. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"7B","shot":"0-shot"}. This record does not assert independent reproduction. ### MPT F1 Score: 38. Result ID: 69911f7f3a6c31484905db45. [Reporting paper]() Recorded evaluation configuration: {"Size":"7B","shot":"0-shot"}. This record does not assert independent reproduction. ### MPT F1 Score: 37.7. Result ID: 69911f7f3a6c31484905db46. [Reporting paper]() Recorded evaluation configuration: {"Size":"7B","shot":"1-shot"}. This record does not assert independent reproduction. ### Llama 1 F1 Score: 36.5. Result ID: 69911f7f3a6c31484905db50. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"13B","shot":"1-shot"}. This record does not assert independent reproduction. ### Llama 1 F1 Score: 32. Result ID: 69911f7f3a6c31484905db4e. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"7B","shot":"1-shot"}. This record does not assert independent reproduction. ### Falcon F1 Score: 24. Result ID: 69911f7f3a6c31484905db49. [Reporting paper]() Recorded evaluation configuration: {"Size":"7B","shot":"0-shot"}. This record does not assert independent reproduction. ### Falcon F1 Score: 18.8. Result ID: 69911f7f3a6c31484905db4a. [Reporting paper]() Recorded evaluation configuration: {"Size":"7B","shot":"1-shot"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.