# Oral Argument Simulation on Oral Argument Evaluation Set 1.0 (test)

Source: [Oral Argument Simulation on Oral Argument Evaluation Set 1.0 (test)](<https://www.sota2.com/research/sota/oral-argument-simulation-on-oral-argument-evaluation-set-1-0-test>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-06T07:48:31.391Z

Evaluation of model performance on oral argument simulation using the Oral Argument Evaluation Set 1.0 (test) dataset, covering metrics such as Overall quality, Adversarial Realism, Human Evaluation, Issue Coverage, Question Type Diversity, and Fallacy Detection.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Oral Argument Simulation |
| Dataset (including any stated split) | Oral Argument Evaluation Set 1.0 (test) |
| Metric | Overall Score |
| Unit | SCALAR |
| Direction | Higher is better |
| Results loaded | 8 of 8 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Overall Score](<https://www.sota2.com/research/sota/oral-argument-simulation-on-oral-argument-evaluation-set-1-0-test.md?metric=Overall%20Score>): Higher is better; unit: SCALAR.

- [Adversarial Realism Score](<https://www.sota2.com/research/sota/oral-argument-simulation-on-oral-argument-evaluation-set-1-0-test.md?metric=Adversarial%20Realism%20Score>): Higher is better; unit: SCALAR.

- [Human Evaluation Score](<https://www.sota2.com/research/sota/oral-argument-simulation-on-oral-argument-evaluation-set-1-0-test.md?metric=Human%20Evaluation%20Score>): Higher is better; unit: SCALAR.

- [Issue Coverage Score](<https://www.sota2.com/research/sota/oral-argument-simulation-on-oral-argument-evaluation-set-1-0-test.md?metric=Issue%20Coverage%20Score>): Higher is better; unit: SCALAR.

- [Question Type Diversity Score](<https://www.sota2.com/research/sota/oral-argument-simulation-on-oral-argument-evaluation-set-1-0-test.md?metric=Question%20Type%20Diversity%20Score>): Higher is better; unit: SCALAR.

- [Fallacy Detection Score](<https://www.sota2.com/research/sota/oral-argument-simulation-on-oral-argument-evaluation-set-1-0-test.md?metric=Fallacy%20Detection%20Score>): Higher is better; unit: SCALAR.

## Recorded results

| Method | Overall Score | Result date | Report paper |
| --- | --- | --- | --- |
| gpt4o | 8 | Not recorded | 69aa5f672d603e6de3e4172b |
| gpt-oss-120b | 7 | Not recorded | 69aa5f672d603e6de3e4172b |
| gpt-oss-120b | 5 | Not recorded | 69aa5f672d603e6de3e4172b |
| Qwen3-32B | 5 | Not recorded | 69aa5f672d603e6de3e4172b |
| gpt4o | 4 | Not recorded | 69aa5f672d603e6de3e4172b |
| Llama-3.3-70B-Instruct | 3 | Not recorded | 69aa5f672d603e6de3e4172b |
| gemini-2.5-pro | 2 | Not recorded | 69aa5f672d603e6de3e4172b |
| gemini-2.5-pro | 1 | Not recorded | 69aa5f672d603e6de3e4172b |

### gpt4o

Overall Score: 8. Result ID: 69aa86296dbb7075b9540540.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f672d603e6de3e4172b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69aa5f672d603e6de3e4172b.md>)

Recorded evaluation configuration: {"Implementation Strategy":"AGENT"}. This record does not assert independent reproduction.

### gpt-oss-120b

Overall Score: 7. Result ID: 69aa86296dbb7075b954053f.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f672d603e6de3e4172b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69aa5f672d603e6de3e4172b.md>)

Recorded evaluation configuration: {"Implementation Strategy":"AGENT"}. This record does not assert independent reproduction.

### gpt-oss-120b

Overall Score: 5. Result ID: 69aa86296dbb7075b954053d.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f672d603e6de3e4172b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69aa5f672d603e6de3e4172b.md>)

Recorded evaluation configuration: {"Implementation Strategy":"PROMPT"}. This record does not assert independent reproduction.

### Qwen3-32B

Overall Score: 5. Result ID: 69aa86296dbb7075b954053e.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f672d603e6de3e4172b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69aa5f672d603e6de3e4172b.md>)

Recorded evaluation configuration: {"Implementation Strategy":"PROMPT"}. This record does not assert independent reproduction.

### gpt4o

Overall Score: 4. Result ID: 69aa86296dbb7075b954053c.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f672d603e6de3e4172b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69aa5f672d603e6de3e4172b.md>)

Recorded evaluation configuration: {"Implementation Strategy":"PROMPT"}. This record does not assert independent reproduction.

### Llama-3.3-70B-Instruct

Overall Score: 3. Result ID: 69aa86296dbb7075b954053b.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f672d603e6de3e4172b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69aa5f672d603e6de3e4172b.md>)

Recorded evaluation configuration: {"Implementation Strategy":"PROMPT"}. This record does not assert independent reproduction.

### gemini-2.5-pro

Overall Score: 2. Result ID: 69aa86296dbb7075b954053a.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f672d603e6de3e4172b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69aa5f672d603e6de3e4172b.md>)

Recorded evaluation configuration: {"Implementation Strategy":"AGENT"}. This record does not assert independent reproduction.

### gemini-2.5-pro

Overall Score: 1. Result ID: 69aa86296dbb7075b9540539.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f672d603e6de3e4172b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69aa5f672d603e6de3e4172b.md>)

Recorded evaluation configuration: {"Implementation Strategy":"PROMPT"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
