# Negotiation performance and belief calibration on CaSiNo native (held-out dialogues)

Source: [Negotiation performance and belief calibration on CaSiNo native (held-out dialogues)](<https://www.sota2.com/research/sota/negotiation-performance-and-belief-calibration-on-casino-native-held-out-dialogues>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-07T08:36:17.885Z

Evaluation of negotiation performance and belief calibration on the native held-out dialogues split of the CaSiNo dataset, utilizing metrics such as Accept-F1, Bid cosine, Strategy macro-F1, and Brier.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Negotiation performance and belief calibration |
| Dataset (including any stated split) | CaSiNo native (held-out dialogues) |
| Metric | Accept-F1 |
| Unit | SCALAR |
| Direction | Higher is better |
| Results loaded | 3 of 3 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Accept-F1](<https://www.sota2.com/research/sota/negotiation-performance-and-belief-calibration-on-casino-native-held-out-dialogues.md?metric=Accept-F1>): Higher is better; unit: SCALAR.

- [Bid Cosine Similarity](<https://www.sota2.com/research/sota/negotiation-performance-and-belief-calibration-on-casino-native-held-out-dialogues.md?metric=Bid%20Cosine%20Similarity>): Higher is better; unit: SCALAR.

- [Macro Strategy F1](<https://www.sota2.com/research/sota/negotiation-performance-and-belief-calibration-on-casino-native-held-out-dialogues.md?metric=Macro%20Strategy%20F1>): Higher is better; unit: SCALAR.

- [Brier Score](<https://www.sota2.com/research/sota/negotiation-performance-and-belief-calibration-on-casino-native-held-out-dialogues.md?metric=Brier%20Score>): Lower is better; unit: SCALAR.

## Recorded results

| Method | Accept-F1 | Result date | Report paper |
| --- | --- | --- | --- |
| 70B structured-CoT | 0.947 | Not recorded | 69fc1cac5497b3b670c88621 |
| Distilled 8B student | 0.908 | Not recorded | 69fc1cac5497b3b670c88621 |
| Bayesian teacher | 0.897 | Not recorded | 69fc1cac5497b3b670c88621 |

### 70B structured-CoT

Accept-F1: 0.947. Result ID: 69fc4e6447f34ce924df056e.

[Reporting paper](<https://www.sota2.com/research/paper/69fc1cac5497b3b670c88621.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa75d11e209131cea6a028.md>)

[Implementation](<https://github.com/meta-llama/llama-agentic-system>)

Recorded evaluation configuration: {"Model scale":"70B","Protocol":"Protocol 3","Self-consistency samples (n)":"5","Temperature":"0.7","Posterior type":"majority-vote shares"}. This record does not assert independent reproduction.

### Distilled 8B student

Accept-F1: 0.908. Result ID: 69fc4e6447f34ce924df0570.

[Reporting paper](<https://www.sota2.com/research/paper/69fc1cac5497b3b670c88621.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69fc1cac5497b3b670c88621.md>)

Recorded evaluation configuration: {"Model scale":"8B","Protocol":"Protocol 3","Posterior type":"native single-pass"}. This record does not assert independent reproduction.

### Bayesian teacher

Accept-F1: 0.897. Result ID: 69fc4e6447f34ce924df056f.

[Reporting paper](<https://www.sota2.com/research/paper/69fc1cac5497b3b670c88621.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69fc1cac5497b3b670c88621.md>)

Recorded evaluation configuration: {"Protocol":"Protocol 3","Posterior type":"native single-pass"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.