# Medical QA Evaluation on A/B testing traffic (online production)

Source: [Medical QA Evaluation on A/B testing traffic (online production)](<https://www.sota2.com/research/sota/medical-qa-evaluation-on-a-b-testing-traffic-online-production>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-26T05:39:38.743Z

Evaluation of a Medical QA system using live A/B testing traffic data, focusing on operational metrics such as user interaction rates and feedback ratios.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Medical QA Evaluation |
| Dataset (including any stated split) | A/B testing traffic (online production) |
| Metric | Completion Rate |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 2 of 2 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Completion Rate](<https://www.sota2.com/research/sota/medical-qa-evaluation-on-a-b-testing-traffic-online-production.md?metric=Completion%20Rate>): Higher is better; unit: PERCENT.

- [Query Reformulation Rate](<https://www.sota2.com/research/sota/medical-qa-evaluation-on-a-b-testing-traffic-online-production.md?metric=Query%20Reformulation%20Rate>): Lower is better; unit: PERCENT.

- [Interaction Rate (UV)](<https://www.sota2.com/research/sota/medical-qa-evaluation-on-a-b-testing-traffic-online-production.md?metric=Interaction%20Rate%20(UV)>): Higher is better; unit: SCALAR.

- [Positive Feedback Ratio (UV)](<https://www.sota2.com/research/sota/medical-qa-evaluation-on-a-b-testing-traffic-online-production.md?metric=Positive%20Feedback%20Ratio%20(UV)>): Higher is better; unit: PERCENT.

## Recorded results

| Method | Completion Rate | Result date | Report paper |
| --- | --- | --- | --- |
| Experimental | 0.7018 | Not recorded | 698eaffec97a0b3e3b5e66a1 |
| Baseline | 0.6396 | Not recorded | 698eaffec97a0b3e3b5e66a1 |

### Experimental

Completion Rate: 0.7018. Result ID: 6995f3785698665ec42695f0.

[Reporting paper](<https://www.sota2.com/research/paper/698eaffec97a0b3e3b5e66a1.md>)

[Method origin paper](<https://www.sota2.com/research/paper/698eaffec97a0b3e3b5e66a1.md>)

Recorded evaluation configuration: {"Group (Bucket)":"Experimental"}. This record does not assert independent reproduction.

### Baseline

Completion Rate: 0.6396. Result ID: 6995f3785698665ec42695ef.

[Reporting paper](<https://www.sota2.com/research/paper/698eaffec97a0b3e3b5e66a1.md>)

Recorded evaluation configuration: {"Group (Bucket)":"Baseline"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
