# Human Preference Evaluation on Oral Argument Simulation (Evaluation set)

Source: [Human Preference Evaluation on Oral Argument Simulation (Evaluation set)](<https://www.sota2.com/research/sota/human-preference-evaluation-on-oral-argument-simulation-evaluation-set>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-06T07:48:31.355Z

Evaluation of model performance based on human preference judgments specifically collected for oral argument simulations, measured by metrics like Win Rate and Bad Rate.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Human Preference Evaluation |
| Dataset (including any stated split) | Human Preference for Oral Argument Simulation (Evaluation set) |
| Metric | Wins |
| Unit | SCALAR |
| Direction | Higher is better |
| Results loaded | 9 of 9 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Wins](<https://www.sota2.com/research/sota/human-preference-evaluation-on-oral-argument-simulation-evaluation-set.md?metric=Wins>): Higher is better; unit: SCALAR.

- [Losses](<https://www.sota2.com/research/sota/human-preference-evaluation-on-oral-argument-simulation-evaluation-set.md?metric=Losses>): Lower is better; unit: SCALAR.

- [Effective Ties](<https://www.sota2.com/research/sota/human-preference-evaluation-on-oral-argument-simulation-evaluation-set.md?metric=Effective%20Ties>): Higher is better; unit: SCALAR.

- [Weighted Win Rate](<https://www.sota2.com/research/sota/human-preference-evaluation-on-oral-argument-simulation-evaluation-set.md?metric=Weighted%20Win%20Rate>): Higher is better; unit: PERCENT.

- [Bad Rate](<https://www.sota2.com/research/sota/human-preference-evaluation-on-oral-argument-simulation-evaluation-set.md?metric=Bad%20Rate>): Lower is better; unit: PERCENT.

## Recorded results

| Method | Wins | Result date | Report paper |
| --- | --- | --- | --- |
| Gemini-2.5-Pro | 72 | Not recorded | 69aa5f672d603e6de3e4172b |
| Llama-3.3-70B-Instruct | 66 | Not recorded | 69aa5f672d603e6de3e4172b |
| GPT-4o | 62 | Not recorded | 69aa5f672d603e6de3e4172b |
| Gemini-2.5-Pro | 62 | Not recorded | 69aa5f672d603e6de3e4172b |
| Ground Truth | 46 | Not recorded | 69aa5f672d603e6de3e4172b |
| GPT-4o | 45 | Not recorded | 69aa5f672d603e6de3e4172b |
| Qwen3-32B | 42 | Not recorded | 69aa5f672d603e6de3e4172b |
| GPT-OSS-120B | 36 | Not recorded | 69aa5f672d603e6de3e4172b |
| GPT-OSS-120B | 24 | Not recorded | 69aa5f672d603e6de3e4172b |

### Gemini-2.5-Pro

Wins: 72. Result ID: 69aa86296dbb7075b9540530.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f672d603e6de3e4172b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69aa5f672d603e6de3e4172b.md>)

Recorded evaluation configuration: {"Simulation Mode":"AGENT"}. This record does not assert independent reproduction.

### Llama-3.3-70B-Instruct

Wins: 66. Result ID: 69aa86296dbb7075b9540531.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f672d603e6de3e4172b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69aa5f672d603e6de3e4172b.md>)

Recorded evaluation configuration: {"Simulation Mode":"PROMPT"}. This record does not assert independent reproduction.

### GPT-4o

Wins: 62. Result ID: 69aa86296dbb7075b9540532.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f672d603e6de3e4172b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69aa5f672d603e6de3e4172b.md>)

Recorded evaluation configuration: {"Simulation Mode":"PROMPT"}. This record does not assert independent reproduction.

### Gemini-2.5-Pro

Wins: 62. Result ID: 69aa86296dbb7075b9540533.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f672d603e6de3e4172b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69aa5f672d603e6de3e4172b.md>)

Recorded evaluation configuration: {"Simulation Mode":"PROMPT"}. This record does not assert independent reproduction.

### Ground Truth

Wins: 46. Result ID: 69aa86296dbb7075b9540534.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f672d603e6de3e4172b.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### GPT-4o

Wins: 45. Result ID: 69aa86296dbb7075b9540535.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f672d603e6de3e4172b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69aa5f672d603e6de3e4172b.md>)

Recorded evaluation configuration: {"Simulation Mode":"AGENT"}. This record does not assert independent reproduction.

### Qwen3-32B

Wins: 42. Result ID: 69aa86296dbb7075b9540536.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f672d603e6de3e4172b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69aa5f672d603e6de3e4172b.md>)

Recorded evaluation configuration: {"Simulation Mode":"PROMPT"}. This record does not assert independent reproduction.

### GPT-OSS-120B

Wins: 36. Result ID: 69aa86296dbb7075b9540537.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f672d603e6de3e4172b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69aa5f672d603e6de3e4172b.md>)

Recorded evaluation configuration: {"Simulation Mode":"PROMPT"}. This record does not assert independent reproduction.

### GPT-OSS-120B

Wins: 24. Result ID: 69aa86296dbb7075b9540538.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f672d603e6de3e4172b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69aa5f672d603e6de3e4172b.md>)

Recorded evaluation configuration: {"Simulation Mode":"AGENT"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
