# Inferring Which Medical Treatments Work from Reports of Clinical Trials

Source: [Inferring Which Medical Treatments Work from Reports of Clinical Trials](<https://www.sota2.com/research/paper/68fa658b1e209131cea0af88>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T21:29:43.441Z

Authors: Eric Lehman, Jay DeYoung, Regina Barzilay, Byron C. Wallace

Published: 2019-04-02T18:17:49.000Z

## Abstract

How do we know if a particular medical treatment actually works? Ideally one would consult all available evidence from relevant clinical trials. Unfortunately, such results are primarily disseminated in natural language scientific articles, imposing substantial burden on those trying to make sense of them. In this paper, we present a new task and corpus for making this unstructured evidence actionable. The task entails inferring reported findings from a full-text article describing a randomized controlled trial (RCT) with respect to a given intervention, comparator, and outcome of interest, e.g., inferring if an article provides evidence supporting the use of aspirin to reduce risk of stroke, as compared to placebo.   We present a new corpus for this task comprising 10,000+ prompts coupled with full-text articles describing RCTs. Results using a suite of models --- ranging from heuristic (rule-based) approaches to attentive neural architectures --- demonstrate the difficulty of the task, which we believe largely owes to the lengthy, technical input texts. To facilitate further work on this important, challenging problem we make the corpus, documentation, a website and leaderboard, and code for baselines and evaluation available at http://evidence-inference.ebm-nlp.com/.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Rationale Extraction on BeerAdvocate Appearance aspect standard (test)](<https://www.sota2.com/research/sota/rationale-extraction-on-beeradvocate-appearance-aspect-standard-test.md>): Rationale Extraction; BeerAdvocate Appearance aspect standard (test); Accuracy: 85.7.

- [Rationale Extraction on BeerAdvocate Aroma aspect standard (test)](<https://www.sota2.com/research/sota/rationale-extraction-on-beeradvocate-aroma-aspect-standard-test.md>): Rationale Extraction; BeerAdvocate Aroma aspect standard (test); Accuracy: 84.2.

- [Rationale Extraction on BeerAdvocate Palate aspect standard (test)](<https://www.sota2.com/research/sota/rationale-extraction-on-beeradvocate-palate-aspect-standard-test.md>): Rationale Extraction; BeerAdvocate Palate aspect standard (test); Accuracy: 0.838.

- [Rationale Extraction on MovieReview](<https://www.sota2.com/research/sota/rationale-extraction-on-moviereview.md>): Rationale Extraction; MovieReview; F1 Score: 24.1.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1904.01606>)

[PDF](<https://arxiv.org/pdf/1904.01606.pdf>)