# Textual Question Answering on bAbI English 10k (test) Source: [Textual Question Answering on bAbI English 10k (test)]() Collection index: [llms.txt]() Data updated: 2026-02-27T06:07:34.913Z Evaluation of model performance on the test split of the bAbI English 10k dataset, focusing on various sub-tasks of textual question answering and reasoning. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Textual Question Answering | | Dataset (including any stated split) | bAbI English 10k (test) | | Metric | Failed Tasks Count (Error \> 5%) | | Unit | SCALAR | | Direction | Lower is better | | Results loaded | 8 of 8 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Failed Tasks Count (Error \> 5%)](): Lower is better; unit: SCALAR. - [Task 2: 2 Supporting Facts](): Higher is better; unit: SCALAR. - [Task 3: 3 Supporting Facts](): Higher is better; unit: SCALAR. - [Task 5: 3 Argument Relations](): Higher is better; unit: SCALAR. - [Task 6: Yes/No Questions](): Higher is better; unit: SCALAR. - [Task 7: Counting](): Higher is better; unit: SCALAR. - [Task 8: Lists/Sets](): Higher is better; unit: SCALAR. - [Task 9: Simple Negation](): Higher is better; unit: SCALAR. - [Task 11: Basic Coreference](): Higher is better; unit: SCALAR. - [Task 14: Time Reasoning](): Higher is better; unit: SCALAR. - [Task 16: Basic Induction](): Higher is better; unit: SCALAR. - [Task 17: Positional Reasoning](): Higher is better; unit: SCALAR. - [Task 18: Size Reasoning](): Higher is better; unit: SCALAR. - [Task 19: Path Finding](): Higher is better; unit: SCALAR. - [Mean Error Rate](): Lower is better; unit: PERCENT. ## Recorded results | Method | Failed Tasks Count (Error \> 5%) | Result date | Report paper | | --- | --- | --- | --- | | QRN | 0 | Not recorded | 68fa631e1e209131ce9fcf34 | | DMN+ | 1 | Not recorded | 68fa631e1e209131ce9fcf34 | | DMN+ | 1 | Not recorded | 68fa62601e209131ce9f8e79 | | DNC | 2 | Not recorded | 68fa631e1e209131ce9fcf34 | | N2N | 3 | Not recorded | 68fa631e1e209131ce9fcf34 | | GMemN2N | 3 | Not recorded | 68fa631e1e209131ce9fcf34 | | E2E | 3 | Not recorded | 68fa62601e209131ce9f8e79 | ### QRN Failed Tasks Count (Error \> 5%): 0. Result ID: 698c6f7137894ef7d9ce5b36. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"layers":6,"reset gate":true,"hidden state dimension":200}. This record does not assert independent reproduction. ### DMN+ Failed Tasks Count (Error \> 5%): 1. Result ID: 698c6f7137894ef7d9ce5b33. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### DMN+ Failed Tasks Count (Error \> 5%): 1. Result ID: 69a1349035dfcabb00e57c9e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### DNC Failed Tasks Count (Error \> 5%): 2. Result ID: 698c6f7137894ef7d9ce5b35. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### N2N Failed Tasks Count (Error \> 5%): 3. Result ID: 698c6f7137894ef7d9ce5b32. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### GMemN2N Failed Tasks Count (Error \> 5%): 3. Result ID: 698c6f7137894ef7d9ce5b34. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### E2E Failed Tasks Count (Error \> 5%): 3. Result ID: 69a1349035dfcabb00e57c9f. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.