# Trajectory Attribution on Who&When n=58 (Hand-Crafted)

Source: [Trajectory Attribution on Who&When n=58 (Hand-Crafted)](<https://www.sota2.com/research/sota/trajectory-attribution-on-who-and-when-n-58-hand-crafted>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-02T09:16:15.178Z

Evaluation of model performance on Trajectory Attribution using Agent-level accuracy and Exact step-level accuracy metrics on the Who&When n=58 (Hand-Crafted) dataset.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Trajectory Attribution |
| Dataset (including any stated split) | Who&When n=58 (Hand-Crafted) |
| Metric | Agent-level Accuracy |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 15 of 15 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Agent-level Accuracy](<https://www.sota2.com/research/sota/trajectory-attribution-on-who-and-when-n-58-hand-crafted.md?metric=Agent-level%20Accuracy>): Higher is better; unit: PERCENT.

- [Exact Step-level Accuracy](<https://www.sota2.com/research/sota/trajectory-attribution-on-who-and-when-n-58-hand-crafted.md?metric=Exact%20Step-level%20Accuracy>): Higher is better; unit: PERCENT.

## Recorded results

| Method | Agent-level Accuracy | Result date | Report paper |
| --- | --- | --- | --- |
| FALAT | 73 | Not recorded | 6a1e6387dbd7150506f85188 |
| FALAT | 66.3 | Not recorded | 6a1e6387dbd7150506f85188 |
| FALAT | 65.5 | Not recorded | 6a1e6387dbd7150506f85188 |
| AgentTracer | 63.8 | Not recorded | 6a1e6387dbd7150506f85188 |
| Direct | 60.4 | Not recorded | 6a1e6387dbd7150506f85188 |
| CHIEF | 55.2 | Not recorded | 6a1e6387dbd7150506f85188 |
| Direct | 53.4 | Not recorded | 6a1e6387dbd7150506f85188 |
| Direct | 51.7 | Not recorded | 6a1e6387dbd7150506f85188 |
| CHIEF | 51 | Not recorded | 6a1e6387dbd7150506f85188 |
| CHIEF | 50 | Not recorded | 6a1e6387dbd7150506f85188 |
| Direct | 50 | Not recorded | 6a1e6387dbd7150506f85188 |
| Direct | 50 | Not recorded | 6a1e6387dbd7150506f85188 |
| Direct | 44.8 | Not recorded | 6a1e6387dbd7150506f85188 |
| Direct | 39.5 | Not recorded | 6a1e6387dbd7150506f85188 |
| Direct | 37.9 | Not recorded | 6a1e6387dbd7150506f85188 |

### FALAT

Agent-level Accuracy: 73. Result ID: 6a1e9e20fbc373592c18fdb1.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6387dbd7150506f85188.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a1e6387dbd7150506f85188.md>)

Recorded evaluation configuration: {"Backbone":"DeepSeek V3.2"}. This record does not assert independent reproduction.

### FALAT

Agent-level Accuracy: 66.3. Result ID: 6a1e9e20fbc373592c18fdb3.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6387dbd7150506f85188.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a1e6387dbd7150506f85188.md>)

Recorded evaluation configuration: {"Backbone":"MiniMax 2.5"}. This record does not assert independent reproduction.

### FALAT

Agent-level Accuracy: 65.5. Result ID: 6a1e9e20fbc373592c18fdaf.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6387dbd7150506f85188.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a1e6387dbd7150506f85188.md>)

Recorded evaluation configuration: {"Backbone":"GPT-5.4-Mini"}. This record does not assert independent reproduction.

### AgentTracer

Agent-level Accuracy: 63.8. Result ID: 6a1e9e20fbc373592c18fdb4.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6387dbd7150506f85188.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7b251e209131cea86d99.md>)

Recorded evaluation configuration: {"Backbone":"AgentTracer-8B"}. This record does not assert independent reproduction.

### Direct

Agent-level Accuracy: 60.4. Result ID: 6a1e9e20fbc373592c18fdb8.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6387dbd7150506f85188.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa790b1e209131cea7bda8.md>)

Recorded evaluation configuration: {"Backbone":"Qwen3-Coder"}. This record does not assert independent reproduction.

### CHIEF

Agent-level Accuracy: 55.2. Result ID: 6a1e9e20fbc373592c18fdae.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6387dbd7150506f85188.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69a5195a2d603e6de3e3c6fd.md>)

Recorded evaluation configuration: {"Backbone":"GPT-5.4-Mini"}. This record does not assert independent reproduction.

### Direct

Agent-level Accuracy: 53.4. Result ID: 6a1e9e20fbc373592c18fdba.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6387dbd7150506f85188.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa790b1e209131cea7bda8.md>)

Recorded evaluation configuration: {"Backbone":"DeepSeek-R1"}. This record does not assert independent reproduction.

### Direct

Agent-level Accuracy: 51.7. Result ID: 6a1e9e20fbc373592c18fdbb.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6387dbd7150506f85188.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa790b1e209131cea7bda8.md>)

Recorded evaluation configuration: {"Backbone":"Gemini-2.5-Pro"}. This record does not assert independent reproduction.

### CHIEF

Agent-level Accuracy: 51. Result ID: 6a1e9e20fbc373592c18fdb2.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6387dbd7150506f85188.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69a5195a2d603e6de3e3c6fd.md>)

Recorded evaluation configuration: {"Backbone":"MiniMax 2.5"}. This record does not assert independent reproduction.

### CHIEF

Agent-level Accuracy: 50. Result ID: 6a1e9e20fbc373592c18fdb0.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6387dbd7150506f85188.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69a5195a2d603e6de3e3c6fd.md>)

Recorded evaluation configuration: {"Backbone":"DeepSeek V3.2"}. This record does not assert independent reproduction.

### Direct

Agent-level Accuracy: 50. Result ID: 6a1e9e20fbc373592c18fdb6.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6387dbd7150506f85188.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa790b1e209131cea7bda8.md>)

Recorded evaluation configuration: {"Backbone":"LLaMA-3.2-3B"}. This record does not assert independent reproduction.

### Direct

Agent-level Accuracy: 50. Result ID: 6a1e9e20fbc373592c18fdbc.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6387dbd7150506f85188.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa790b1e209131cea7bda8.md>)

Recorded evaluation configuration: {"Backbone":"Claude-Sonnet-4"}. This record does not assert independent reproduction.

### Direct

Agent-level Accuracy: 44.8. Result ID: 6a1e9e20fbc373592c18fdb7.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6387dbd7150506f85188.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa790b1e209131cea7bda8.md>)

Recorded evaluation configuration: {"Backbone":"Qwen3-32B"}. This record does not assert independent reproduction.

### Direct

Agent-level Accuracy: 39.5. Result ID: 6a1e9e20fbc373592c18fdb5.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6387dbd7150506f85188.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa790b1e209131cea7bda8.md>)

Recorded evaluation configuration: {"Backbone":"Qwen3-8B"}. This record does not assert independent reproduction.

### Direct

Agent-level Accuracy: 37.9. Result ID: 6a1e9e20fbc373592c18fdb9.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6387dbd7150506f85188.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa790b1e209131cea7bda8.md>)

Recorded evaluation configuration: {"Backbone":"GPT-4.1"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
