# Natural language generation on Inspect

Source: [Natural language generation on Inspect](<https://www.sota2.com/research/sota/natural-language-generation-on-inspect>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-01T10:15:07.003Z

Evaluation of Natural Language Generation performance on the Inspect dataset.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Natural language generation |
| Dataset (including any stated split) | Inspect |
| Metric | BLEU-1 |
| Unit | SCALAR |
| Direction | Higher is better |
| Results loaded | 14 of 14 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [BLEU-1](<https://www.sota2.com/research/sota/natural-language-generation-on-inspect.md?metric=BLEU-1>): Higher is better; unit: SCALAR.

- [BLEU-2](<https://www.sota2.com/research/sota/natural-language-generation-on-inspect.md?metric=BLEU-2>): Higher is better; unit: SCALAR.

- [BLEU-3](<https://www.sota2.com/research/sota/natural-language-generation-on-inspect.md?metric=BLEU-3>): Higher is better; unit: SCALAR.

- [BLEU-4](<https://www.sota2.com/research/sota/natural-language-generation-on-inspect.md?metric=BLEU-4>): Higher is better; unit: SCALAR.

- [ROUGE-L](<https://www.sota2.com/research/sota/natural-language-generation-on-inspect.md?metric=ROUGE-L>): Higher is better; unit: SCALAR.

- [METEOR](<https://www.sota2.com/research/sota/natural-language-generation-on-inspect.md?metric=METEOR>): Higher is better; unit: SCALAR.

## Recorded results

| Method | BLEU-1 | Result date | Report paper |
| --- | --- | --- | --- |
| Astra | 0.4622 | Not recorded | 6a1d1237dbd7150506f84193 |
| R2GenGPT | 0.4488 | Not recorded | 6a1d1237dbd7150506f84193 |
| M3D | 0.4314 | Not recorded | 6a1d1237dbd7150506f84193 |
| RadFM | 0.4017 | Not recorded | 6a1d1237dbd7150506f84193 |
| MedGemma | 0.3974 | Not recorded | 6a1d1237dbd7150506f84193 |
| Gemini-3 | 0.3954 | Not recorded | 6a1d1237dbd7150506f84193 |
| HuluMed | 0.375 | Not recorded | 6a1d1237dbd7150506f84193 |
| LLaVA | 0.359 | Not recorded | 6a1d1237dbd7150506f84193 |
| Qwen3-VL | 0.3549 | Not recorded | 6a1d1237dbd7150506f84193 |
| Lingshu | 0.3408 | Not recorded | 6a1d1237dbd7150506f84193 |
| Lingshu | 0.3036 | Not recorded | 6a1d1237dbd7150506f84193 |
| MedGemma | 0.2954 | Not recorded | 6a1d1237dbd7150506f84193 |
| HuluMed | 0.28 | Not recorded | 6a1d1237dbd7150506f84193 |
| HuluMed | 0.2623 | Not recorded | 6a1d1237dbd7150506f84193 |

### Astra

BLEU-1: 0.4622. Result ID: 6a1d5b0dfbc373592c172aa6.

[Reporting paper](<https://www.sota2.com/research/paper/6a1d1237dbd7150506f84193.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a1d1237dbd7150506f84193.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### R2GenGPT

BLEU-1: 0.4488. Result ID: 6a1d5b0dfbc373592c172ab2.

[Reporting paper](<https://www.sota2.com/research/paper/6a1d1237dbd7150506f84193.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa71211e209131cea4f5fa.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### M3D

BLEU-1: 0.4314. Result ID: 6a1d5b0dfbc373592c172aac.

[Reporting paper](<https://www.sota2.com/research/paper/6a1d1237dbd7150506f84193.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72be1e209131cea58507.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### RadFM

BLEU-1: 0.4017. Result ID: 6a1d5b0dfbc373592c172aad.

[Reporting paper](<https://www.sota2.com/research/paper/6a1d1237dbd7150506f84193.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa71411e209131cea5023d.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### MedGemma

BLEU-1: 0.3974. Result ID: 6a1d5b0dfbc373592c172ab0.

[Reporting paper](<https://www.sota2.com/research/paper/6a1d1237dbd7150506f84193.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa79ff1e209131cea808a4.md>)

Recorded evaluation configuration: {"Parameters":"27B"}. This record does not assert independent reproduction.

### Gemini-3

BLEU-1: 0.3954. Result ID: 6a1d5b0dfbc373592c172aa7.

[Reporting paper](<https://www.sota2.com/research/paper/6a1d1237dbd7150506f84193.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78731e209131cea78622.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### HuluMed

BLEU-1: 0.375. Result ID: 6a1d5b0dfbc373592c172aaa.

[Reporting paper](<https://www.sota2.com/research/paper/6a1d1237dbd7150506f84193.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7bfd1e209131cea8bb56.md>)

Recorded evaluation configuration: {"Parameters":"14B"}. This record does not assert independent reproduction.

### LLaVA

BLEU-1: 0.359. Result ID: 6a1d5b0dfbc373592c172ab3.

[Reporting paper](<https://www.sota2.com/research/paper/6a1d1237dbd7150506f84193.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### Qwen3-VL

BLEU-1: 0.3549. Result ID: 6a1d5b0dfbc373592c172aa8.

[Reporting paper](<https://www.sota2.com/research/paper/6a1d1237dbd7150506f84193.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa788e1e209131cea79116.md>)

Recorded evaluation configuration: {"Parameters":"8B"}. This record does not assert independent reproduction.

### Lingshu

BLEU-1: 0.3408. Result ID: 6a1d5b0dfbc373592c172aaf.

[Reporting paper](<https://www.sota2.com/research/paper/6a1d1237dbd7150506f84193.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa795a1e209131cea7d387.md>)

Recorded evaluation configuration: {"Parameters":"7B"}. This record does not assert independent reproduction.

### Lingshu

BLEU-1: 0.3036. Result ID: 6a1d5b0dfbc373592c172aae.

[Reporting paper](<https://www.sota2.com/research/paper/6a1d1237dbd7150506f84193.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa795a1e209131cea7d387.md>)

Recorded evaluation configuration: {"Parameters":"32B"}. This record does not assert independent reproduction.

### MedGemma

BLEU-1: 0.2954. Result ID: 6a1d5b0dfbc373592c172ab1.

[Reporting paper](<https://www.sota2.com/research/paper/6a1d1237dbd7150506f84193.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa79ff1e209131cea808a4.md>)

Recorded evaluation configuration: {"Parameters":"4B"}. This record does not assert independent reproduction.

### HuluMed

BLEU-1: 0.28. Result ID: 6a1d5b0dfbc373592c172aab.

[Reporting paper](<https://www.sota2.com/research/paper/6a1d1237dbd7150506f84193.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7bfd1e209131cea8bb56.md>)

Recorded evaluation configuration: {"Parameters":"7B"}. This record does not assert independent reproduction.

### HuluMed

BLEU-1: 0.2623. Result ID: 6a1d5b0dfbc373592c172aa9.

[Reporting paper](<https://www.sota2.com/research/paper/6a1d1237dbd7150506f84193.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7bfd1e209131cea8bb56.md>)

Recorded evaluation configuration: {"Parameters":"32B"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
