# Tracking Shuffled Objects on Tracking Shuffled Objects 5 objects (test) Source: [Tracking Shuffled Objects on Tracking Shuffled Objects 5 objects (test)]() Collection index: [llms.txt]() Data updated: 2026-05-21T13:15:37.564Z Evaluation of model performance on the Tracking Shuffled Objects task using the 5-object test set, measured by accuracy. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Tracking Shuffled Objects | | Dataset (including any stated split) | Tracking Shuffled Objects 5 objects (test) | | Metric | Accuracy (TSO 5-obj) | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 16 of 16 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Accuracy (TSO 5-obj)](): Higher is better; unit: PERCENT. ## Recorded results | Method | Accuracy (TSO 5-obj) | Result date | Report paper | | --- | --- | --- | --- | | TextReg | 94.1 | Not recorded | 6a0e91c1f984739ed6738d75 | | CoT | 89.6 | Not recorded | 6a0e91c1f984739ed6738d75 | | REVOLVE | 86.8 | Not recorded | 6a0e91c1f984739ed6738d75 | | TextGrad | 83.6 | Not recorded | 6a0e91c1f984739ed6738d75 | | TextReg | 79.7 | Not recorded | 6a0e91c1f984739ed6738d75 | | REVOLVE | 76 | Not recorded | 6a0e91c1f984739ed6738d75 | | TextGrad | 69.7 | Not recorded | 6a0e91c1f984739ed6738d75 | | CoT | 58.8 | Not recorded | 6a0e91c1f984739ed6738d75 | | TextReg | 54.3 | Not recorded | 6a0e91c1f984739ed6738d75 | | REVOLVE | 46.1 | Not recorded | 6a0e91c1f984739ed6738d75 | | TextGrad | 45.9 | Not recorded | 6a0e91c1f984739ed6738d75 | | TextReg | 45.4 | Not recorded | 6a0e91c1f984739ed6738d75 | | CoT | 42 | Not recorded | 6a0e91c1f984739ed6738d75 | | REVOLVE | 40.3 | Not recorded | 6a0e91c1f984739ed6738d75 | | TextGrad | 38.2 | Not recorded | 6a0e91c1f984739ed6738d75 | | CoT | 35.6 | Not recorded | 6a0e91c1f984739ed6738d75 | ### TextReg Accuracy (TSO 5-obj): 94.1. Result ID: 6a0f038ffbc373592c07c21c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Test Engine":"Phi-3.5-Mini-Instruct"}. This record does not assert independent reproduction. ### CoT Accuracy (TSO 5-obj): 89.6. Result ID: 6a0f038ffbc373592c07c219. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Test Engine":"Phi-3.5-Mini-Instruct"}. This record does not assert independent reproduction. ### REVOLVE Accuracy (TSO 5-obj): 86.8. Result ID: 6a0f038ffbc373592c07c21b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Test Engine":"Phi-3.5-Mini-Instruct"}. This record does not assert independent reproduction. ### TextGrad Accuracy (TSO 5-obj): 83.6. Result ID: 6a0f038ffbc373592c07c21a. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Test Engine":"Phi-3.5-Mini-Instruct"}. This record does not assert independent reproduction. ### TextReg Accuracy (TSO 5-obj): 79.7. Result ID: 6a0f038ffbc373592c07c214. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Test Engine":"Llama-3.1-8B-Instruct"}. This record does not assert independent reproduction. ### REVOLVE Accuracy (TSO 5-obj): 76. Result ID: 6a0f038ffbc373592c07c213. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Test Engine":"Llama-3.1-8B-Instruct"}. This record does not assert independent reproduction. ### TextGrad Accuracy (TSO 5-obj): 69.7. Result ID: 6a0f038ffbc373592c07c212. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Test Engine":"Llama-3.1-8B-Instruct"}. This record does not assert independent reproduction. ### CoT Accuracy (TSO 5-obj): 58.8. Result ID: 6a0f038ffbc373592c07c211. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Test Engine":"Llama-3.1-8B-Instruct"}. This record does not assert independent reproduction. ### TextReg Accuracy (TSO 5-obj): 54.3. Result ID: 6a0f038ffbc373592c07c218. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Test Engine":"Llama-3-8B-Instruct"}. This record does not assert independent reproduction. ### REVOLVE Accuracy (TSO 5-obj): 46.1. Result ID: 6a0f038ffbc373592c07c217. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Test Engine":"Llama-3-8B-Instruct"}. This record does not assert independent reproduction. ### TextGrad Accuracy (TSO 5-obj): 45.9. Result ID: 6a0f038ffbc373592c07c216. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Test Engine":"Llama-3-8B-Instruct"}. This record does not assert independent reproduction. ### TextReg Accuracy (TSO 5-obj): 45.4. Result ID: 6a0f038ffbc373592c07c210. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Test Engine":"Qwen2-7B-Instruct"}. This record does not assert independent reproduction. ### CoT Accuracy (TSO 5-obj): 42. Result ID: 6a0f038ffbc373592c07c20d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Test Engine":"Qwen2-7B-Instruct"}. This record does not assert independent reproduction. ### REVOLVE Accuracy (TSO 5-obj): 40.3. Result ID: 6a0f038ffbc373592c07c20f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Test Engine":"Qwen2-7B-Instruct"}. This record does not assert independent reproduction. ### TextGrad Accuracy (TSO 5-obj): 38.2. Result ID: 6a0f038ffbc373592c07c20e. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Test Engine":"Qwen2-7B-Instruct"}. This record does not assert independent reproduction. ### CoT Accuracy (TSO 5-obj): 35.6. Result ID: 6a0f038ffbc373592c07c215. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Test Engine":"Llama-3-8B-Instruct"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.