# Simulation Task Planning on BEHAVIOR-1K 15 tasks

Source: [Simulation Task Planning on BEHAVIOR-1K 15 tasks](<https://www.sota2.com/research/sota/simulation-task-planning-on-behavior-1k-15-tasks>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-09T08:06:38.682Z

Evaluation of Simulation Task Planning performance on the BEHAVIOR-1K 15 tasks subset, measured by BT Valid, SR, and Pass@3.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Simulation Task Planning |
| Dataset (including any stated split) | BEHAVIOR-1K 15 tasks |
| Metric | BT Valid |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 14 of 14 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [BT Valid](<https://www.sota2.com/research/sota/simulation-task-planning-on-behavior-1k-15-tasks.md?metric=BT%20Valid>): Higher is better; unit: PERCENT.

- [Success Rate (SR)](<https://www.sota2.com/research/sota/simulation-task-planning-on-behavior-1k-15-tasks.md?metric=Success%20Rate%20(SR)>): Higher is better; unit: PERCENT.

- [Pass@3](<https://www.sota2.com/research/sota/simulation-task-planning-on-behavior-1k-15-tasks.md?metric=Pass%403>): Higher is better; unit: PERCENT.

## Recorded results

| Method | BT Valid | Result date | Report paper |
| --- | --- | --- | --- |
| GPT-5 | 100 | Not recorded | 69ae53df2d603e6de3e428ae |
| GPT-5 | 100 | Not recorded | 69ae53df2d603e6de3e428ae |
| Gemma-3 4B | 100 | Not recorded | 69ae53df2d603e6de3e428ae |
| Gemma-3 4B | 100 | Not recorded | 69ae53df2d603e6de3e428ae |
| Qwen2.5-VL | 100 | Not recorded | 69ae53df2d603e6de3e428ae |
| Qwen2.5-VL | 100 | Not recorded | 69ae53df2d603e6de3e428ae |
| SmolVLM2 | 27 | Not recorded | 69ae53df2d603e6de3e428ae |
| SmolVLM2 | 7 | Not recorded | 69ae53df2d603e6de3e428ae |
| Gemma-3 4B | 0 | Not recorded | 69ae53df2d603e6de3e428ae |
| Gemma-3 4B | 0 | Not recorded | 69ae53df2d603e6de3e428ae |
| Qwen2.5-VL | 0 | Not recorded | 69ae53df2d603e6de3e428ae |
| Qwen2.5-VL | 0 | Not recorded | 69ae53df2d603e6de3e428ae |
| SmolVLM2 | 0 | Not recorded | 69ae53df2d603e6de3e428ae |
| SmolVLM2 | 0 | Not recorded | 69ae53df2d603e6de3e428ae |

### GPT-5

BT Valid: 100. Result ID: 69ae7f7b6dbb7075b9567a0e.

[Reporting paper](<https://www.sota2.com/research/paper/69ae53df2d603e6de3e428ae.md>)

Recorded evaluation configuration: {"Prompting":"CoT"}. This record does not assert independent reproduction.

### GPT-5

BT Valid: 100. Result ID: 69ae7f7b6dbb7075b9567a0f.

[Reporting paper](<https://www.sota2.com/research/paper/69ae53df2d603e6de3e428ae.md>)

Recorded evaluation configuration: {"Prompting":"ZS"}. This record does not assert independent reproduction.

### Gemma-3 4B

BT Valid: 100. Result ID: 69ae7f7b6dbb7075b9567a10.

[Reporting paper](<https://www.sota2.com/research/paper/69ae53df2d603e6de3e428ae.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69ae53df2d603e6de3e428ae.md>)

Recorded evaluation configuration: {"Var.":"FT","Prompting":"CoT"}. This record does not assert independent reproduction.

### Gemma-3 4B

BT Valid: 100. Result ID: 69ae7f7b6dbb7075b9567a11.

[Reporting paper](<https://www.sota2.com/research/paper/69ae53df2d603e6de3e428ae.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69ae53df2d603e6de3e428ae.md>)

Recorded evaluation configuration: {"Var.":"FT","Prompting":"ZS"}. This record does not assert independent reproduction.

### Qwen2.5-VL

BT Valid: 100. Result ID: 69ae7f7b6dbb7075b9567a14.

[Reporting paper](<https://www.sota2.com/research/paper/69ae53df2d603e6de3e428ae.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69ae53df2d603e6de3e428ae.md>)

Recorded evaluation configuration: {"Var.":"FT","Prompting":"CoT"}. This record does not assert independent reproduction.

### Qwen2.5-VL

BT Valid: 100. Result ID: 69ae7f7b6dbb7075b9567a15.

[Reporting paper](<https://www.sota2.com/research/paper/69ae53df2d603e6de3e428ae.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69ae53df2d603e6de3e428ae.md>)

Recorded evaluation configuration: {"Var.":"FT","Prompting":"ZS"}. This record does not assert independent reproduction.

### SmolVLM2

BT Valid: 27. Result ID: 69ae7f7b6dbb7075b9567a19.

[Reporting paper](<https://www.sota2.com/research/paper/69ae53df2d603e6de3e428ae.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69ae53df2d603e6de3e428ae.md>)

Recorded evaluation configuration: {"Var.":"FT","Prompting":"ZS"}. This record does not assert independent reproduction.

### SmolVLM2

BT Valid: 7. Result ID: 69ae7f7b6dbb7075b9567a18.

[Reporting paper](<https://www.sota2.com/research/paper/69ae53df2d603e6de3e428ae.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69ae53df2d603e6de3e428ae.md>)

Recorded evaluation configuration: {"Var.":"FT","Prompting":"CoT"}. This record does not assert independent reproduction.

### Gemma-3 4B

BT Valid: 0. Result ID: 69ae7f7b6dbb7075b9567a12.

[Reporting paper](<https://www.sota2.com/research/paper/69ae53df2d603e6de3e428ae.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69ae53df2d603e6de3e428ae.md>)

Recorded evaluation configuration: {"Var.":"Base","Prompting":"CoT"}. This record does not assert independent reproduction.

### Gemma-3 4B

BT Valid: 0. Result ID: 69ae7f7b6dbb7075b9567a13.

[Reporting paper](<https://www.sota2.com/research/paper/69ae53df2d603e6de3e428ae.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69ae53df2d603e6de3e428ae.md>)

Recorded evaluation configuration: {"Var.":"Base","Prompting":"ZS"}. This record does not assert independent reproduction.

### Qwen2.5-VL

BT Valid: 0. Result ID: 69ae7f7b6dbb7075b9567a16.

[Reporting paper](<https://www.sota2.com/research/paper/69ae53df2d603e6de3e428ae.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69ae53df2d603e6de3e428ae.md>)

Recorded evaluation configuration: {"Var.":"Base","Prompting":"CoT"}. This record does not assert independent reproduction.

### Qwen2.5-VL

BT Valid: 0. Result ID: 69ae7f7b6dbb7075b9567a17.

[Reporting paper](<https://www.sota2.com/research/paper/69ae53df2d603e6de3e428ae.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69ae53df2d603e6de3e428ae.md>)

Recorded evaluation configuration: {"Var.":"Base","Prompting":"ZS"}. This record does not assert independent reproduction.

### SmolVLM2

BT Valid: 0. Result ID: 69ae7f7b6dbb7075b9567a1a.

[Reporting paper](<https://www.sota2.com/research/paper/69ae53df2d603e6de3e428ae.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69ae53df2d603e6de3e428ae.md>)

Recorded evaluation configuration: {"Var.":"Base","Prompting":"CoT"}. This record does not assert independent reproduction.

### SmolVLM2

BT Valid: 0. Result ID: 69ae7f7b6dbb7075b9567a1b.

[Reporting paper](<https://www.sota2.com/research/paper/69ae53df2d603e6de3e428ae.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69ae53df2d603e6de3e428ae.md>)

Recorded evaluation configuration: {"Var.":"Base","Prompting":"ZS"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
