# Skill invocation in LLM agents on 30 artifact-rich tasks (test)

Source: [Skill invocation in LLM agents on 30 artifact-rich tasks (test)](<https://www.sota2.com/research/sota/skill-invocation-in-llm-agents-on-30-artifact-rich-tasks-test>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-04T17:05:04.765Z

Evaluation of LLM agent performance on skill invocation across 30 artifact-rich tasks, measured by win/loss/tie counts for specific skill categories (Data Computation, Document Creation, Motion Video, Visual Design, Web Interaction) and an overall Bradley-Terry score.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Skill invocation in LLM agents |
| Dataset (including any stated split) | 30 artifact-rich tasks (test) |
| Metric | Data Computation Wins |
| Unit | SCALAR |
| Direction | Higher is better |
| Results loaded | 15 of 15 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Data Computation Wins](<https://www.sota2.com/research/sota/skill-invocation-in-llm-agents-on-30-artifact-rich-tasks-test.md?metric=Data%20Computation%20Wins>): Higher is better; unit: SCALAR.

- [Data Computation Ties](<https://www.sota2.com/research/sota/skill-invocation-in-llm-agents-on-30-artifact-rich-tasks-test.md?metric=Data%20Computation%20Ties>): Higher is better; unit: SCALAR.

- [Data Computation Losses](<https://www.sota2.com/research/sota/skill-invocation-in-llm-agents-on-30-artifact-rich-tasks-test.md?metric=Data%20Computation%20Losses>): Lower is better; unit: SCALAR.

- [Document Creation Wins](<https://www.sota2.com/research/sota/skill-invocation-in-llm-agents-on-30-artifact-rich-tasks-test.md?metric=Document%20Creation%20Wins>): Higher is better; unit: SCALAR.

- [Document Creation Ties](<https://www.sota2.com/research/sota/skill-invocation-in-llm-agents-on-30-artifact-rich-tasks-test.md?metric=Document%20Creation%20Ties>): Higher is better; unit: SCALAR.

- [Document Creation Losses](<https://www.sota2.com/research/sota/skill-invocation-in-llm-agents-on-30-artifact-rich-tasks-test.md?metric=Document%20Creation%20Losses>): Lower is better; unit: SCALAR.

- [Motion Video Wins](<https://www.sota2.com/research/sota/skill-invocation-in-llm-agents-on-30-artifact-rich-tasks-test.md?metric=Motion%20Video%20Wins>): Higher is better; unit: SCALAR.

- [Motion Video Ties](<https://www.sota2.com/research/sota/skill-invocation-in-llm-agents-on-30-artifact-rich-tasks-test.md?metric=Motion%20Video%20Ties>): Higher is better; unit: SCALAR.

- [Motion Video Losses](<https://www.sota2.com/research/sota/skill-invocation-in-llm-agents-on-30-artifact-rich-tasks-test.md?metric=Motion%20Video%20Losses>): Lower is better; unit: SCALAR.

- [Visual Design Wins](<https://www.sota2.com/research/sota/skill-invocation-in-llm-agents-on-30-artifact-rich-tasks-test.md?metric=Visual%20Design%20Wins>): Higher is better; unit: SCALAR.

- [Visual Design Ties](<https://www.sota2.com/research/sota/skill-invocation-in-llm-agents-on-30-artifact-rich-tasks-test.md?metric=Visual%20Design%20Ties>): Higher is better; unit: SCALAR.

- [Visual Design Losses](<https://www.sota2.com/research/sota/skill-invocation-in-llm-agents-on-30-artifact-rich-tasks-test.md?metric=Visual%20Design%20Losses>): Lower is better; unit: SCALAR.

- [Web Interaction Wins](<https://www.sota2.com/research/sota/skill-invocation-in-llm-agents-on-30-artifact-rich-tasks-test.md?metric=Web%20Interaction%20Wins>): Higher is better; unit: SCALAR.

- [Web Interaction Ties](<https://www.sota2.com/research/sota/skill-invocation-in-llm-agents-on-30-artifact-rich-tasks-test.md?metric=Web%20Interaction%20Ties>): Higher is better; unit: SCALAR.

- [Web Interaction Losses](<https://www.sota2.com/research/sota/skill-invocation-in-llm-agents-on-30-artifact-rich-tasks-test.md?metric=Web%20Interaction%20Losses>): Lower is better; unit: SCALAR.

- [Bradley-Terry Score](<https://www.sota2.com/research/sota/skill-invocation-in-llm-agents-on-30-artifact-rich-tasks-test.md?metric=Bradley-Terry%20Score>): Higher is better; unit: SCALAR.

## Recorded results

| Method | Data Computation Wins | Result date | Report paper |
| --- | --- | --- | --- |
| Quality-First | 18 | Not recorded | 69a66c582d603e6de3e3d194 |
| Quality-First | 15 | Not recorded | 69a66c582d603e6de3e3d194 |
| Quality-First | 14 | Not recorded | 69a66c582d603e6de3e3d194 |
| Efficiency-First | 13 | Not recorded | 69a66c582d603e6de3e3d194 |
| w/ Full Pool. | 11 | Not recorded | 69a66c582d603e6de3e3d194 |
| Efficiency-First | 11 | Not recorded | 69a66c582d603e6de3e3d194 |
| Simplicity-First | 10 | Not recorded | 69a66c582d603e6de3e3d194 |
| Efficiency-First | 10 | Not recorded | 69a66c582d603e6de3e3d194 |
| Simplicity-First | 9 | Not recorded | 69a66c582d603e6de3e3d194 |
| Vanilla | 9 | Not recorded | 69a66c582d603e6de3e3d194 |
| Simplicity-First | 8 | Not recorded | 69a66c582d603e6de3e3d194 |
| w/ Full Pool. | 7 | Not recorded | 69a66c582d603e6de3e3d194 |
| Vanilla | 6 | Not recorded | 69a66c582d603e6de3e3d194 |
| Vanilla | 6 | Not recorded | 69a66c582d603e6de3e3d194 |
| w/ Full Pool. | 5 | Not recorded | 69a66c582d603e6de3e3d194 |

### Quality-First

Data Computation Wins: 18. Result ID: 69a866316dbb7075b9501be8.

[Reporting paper](<https://www.sota2.com/research/paper/69a66c582d603e6de3e3d194.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69a66c582d603e6de3e3d194.md>)

Recorded evaluation configuration: {"Ecosystem Size (\|S\|)":"200"}. This record does not assert independent reproduction.

### Quality-First

Data Computation Wins: 15. Result ID: 69a866316dbb7075b9501bf2.

[Reporting paper](<https://www.sota2.com/research/paper/69a66c582d603e6de3e3d194.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69a66c582d603e6de3e3d194.md>)

Recorded evaluation configuration: {"Ecosystem Size (\|S\|)":"200K"}. This record does not assert independent reproduction.

### Quality-First

Data Computation Wins: 14. Result ID: 69a866316dbb7075b9501bed.

[Reporting paper](<https://www.sota2.com/research/paper/69a66c582d603e6de3e3d194.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69a66c582d603e6de3e3d194.md>)

Recorded evaluation configuration: {"Ecosystem Size (\|S\|)":"1K"}. This record does not assert independent reproduction.

### Efficiency-First

Data Computation Wins: 13. Result ID: 69a866316dbb7075b9501be9.

[Reporting paper](<https://www.sota2.com/research/paper/69a66c582d603e6de3e3d194.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69a66c582d603e6de3e3d194.md>)

Recorded evaluation configuration: {"Ecosystem Size (\|S\|)":"200"}. This record does not assert independent reproduction.

### w/ Full Pool.

Data Computation Wins: 11. Result ID: 69a866316dbb7075b9501bf0.

[Reporting paper](<https://www.sota2.com/research/paper/69a66c582d603e6de3e3d194.md>)

Recorded evaluation configuration: {"Ecosystem Size (\|S\|)":"1K"}. This record does not assert independent reproduction.

### Efficiency-First

Data Computation Wins: 11. Result ID: 69a866316dbb7075b9501bf3.

[Reporting paper](<https://www.sota2.com/research/paper/69a66c582d603e6de3e3d194.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69a66c582d603e6de3e3d194.md>)

Recorded evaluation configuration: {"Ecosystem Size (\|S\|)":"200K"}. This record does not assert independent reproduction.

### Simplicity-First

Data Computation Wins: 10. Result ID: 69a866316dbb7075b9501bea.

[Reporting paper](<https://www.sota2.com/research/paper/69a66c582d603e6de3e3d194.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69a66c582d603e6de3e3d194.md>)

Recorded evaluation configuration: {"Ecosystem Size (\|S\|)":"200"}. This record does not assert independent reproduction.

### Efficiency-First

Data Computation Wins: 10. Result ID: 69a866316dbb7075b9501bee.

[Reporting paper](<https://www.sota2.com/research/paper/69a66c582d603e6de3e3d194.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69a66c582d603e6de3e3d194.md>)

Recorded evaluation configuration: {"Ecosystem Size (\|S\|)":"1K"}. This record does not assert independent reproduction.

### Simplicity-First

Data Computation Wins: 9. Result ID: 69a866316dbb7075b9501bef.

[Reporting paper](<https://www.sota2.com/research/paper/69a66c582d603e6de3e3d194.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69a66c582d603e6de3e3d194.md>)

Recorded evaluation configuration: {"Ecosystem Size (\|S\|)":"1K"}. This record does not assert independent reproduction.

### Vanilla

Data Computation Wins: 9. Result ID: 69a866316dbb7075b9501bf1.

[Reporting paper](<https://www.sota2.com/research/paper/69a66c582d603e6de3e3d194.md>)

Recorded evaluation configuration: {"Ecosystem Size (\|S\|)":"1K"}. This record does not assert independent reproduction.

### Simplicity-First

Data Computation Wins: 8. Result ID: 69a866316dbb7075b9501bf4.

[Reporting paper](<https://www.sota2.com/research/paper/69a66c582d603e6de3e3d194.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69a66c582d603e6de3e3d194.md>)

Recorded evaluation configuration: {"Ecosystem Size (\|S\|)":"200K"}. This record does not assert independent reproduction.

### w/ Full Pool.

Data Computation Wins: 7. Result ID: 69a866316dbb7075b9501bf5.

[Reporting paper](<https://www.sota2.com/research/paper/69a66c582d603e6de3e3d194.md>)

Recorded evaluation configuration: {"Ecosystem Size (\|S\|)":"200K"}. This record does not assert independent reproduction.

### Vanilla

Data Computation Wins: 6. Result ID: 69a866316dbb7075b9501bec.

[Reporting paper](<https://www.sota2.com/research/paper/69a66c582d603e6de3e3d194.md>)

Recorded evaluation configuration: {"Ecosystem Size (\|S\|)":"200"}. This record does not assert independent reproduction.

### Vanilla

Data Computation Wins: 6. Result ID: 69a866316dbb7075b9501bf6.

[Reporting paper](<https://www.sota2.com/research/paper/69a66c582d603e6de3e3d194.md>)

Recorded evaluation configuration: {"Ecosystem Size (\|S\|)":"200K"}. This record does not assert independent reproduction.

### w/ Full Pool.

Data Computation Wins: 5. Result ID: 69a866316dbb7075b9501beb.

[Reporting paper](<https://www.sota2.com/research/paper/69a66c582d603e6de3e3d194.md>)

Recorded evaluation configuration: {"Ecosystem Size (\|S\|)":"200"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
