# Long-horizon language-conditioned manipulation on Calvin ABC→D Source: [Long-horizon language-conditioned manipulation on Calvin ABC→D]() Collection index: [llms.txt]() Data updated: 2026-06-04T07:05:29.781Z Evaluation of model performance on the Calvin ABC→D dataset for long-horizon, language-conditioned manipulation, measured by the success rate of completing 1 through 5 tasks sequentially and the average length of successful sequences. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Long-horizon language-conditioned manipulation | | Dataset (including any stated split) | Calvin ABC→D | | Metric | Average Sequence Length | | Unit | SCALAR | | Direction | Higher is better | | Results loaded | 18 of 18 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Average Sequence Length](): Higher is better; unit: SCALAR. - [Success Rate (Seq 1)](): Higher is better; unit: PERCENT. - [Success Rate (Seq 2)](): Higher is better; unit: PERCENT. - [Success Rate (Seq 3)](): Higher is better; unit: PERCENT. - [Success Rate (Seq 4)](): Higher is better; unit: PERCENT. - [Success Rate (Seq 5)](): Higher is better; unit: PERCENT. ## Recorded results | Method | Average Sequence Length | Result date | Report paper | | --- | --- | --- | --- | | X-VLA | 4.43 | Not recorded | 6a21067bc10484bac23efff5 | | VLA-Adapter | 4.42 | Not recorded | 6a21067bc10484bac23efff5 | | UniJEPA | 4.11 | Not recorded | 68fa7c0b1e209131cea8c0d9 | | UP-VLA\* | 4.08 | Not recorded | 68fa7c0b1e209131cea8c0d9 | | S2 | 3.95 | Not recorded | 6a21067bc10484bac23efff5 | | π0.5 | 3.92 | Not recorded | 6a21067bc10484bac23efff5 | | UniVLA | 3.8 | Not recorded | 6a21067bc10484bac23efff5 | | Lie Diffuser Actor | 3.584 | Not recorded | 6a1e6438dbd7150506f85881 | | VPP\* | 3.58 | Not recorded | 68fa7c0b1e209131cea8c0d9 | | Lie Diffuser Actor | 3.512 | Not recorded | 6a1e6438dbd7150506f85881 | | Lie Diffuser Actor | 3.474 | Not recorded | 6a1e6438dbd7150506f85881 | | Lie Diffuser Actor | 3.404 | Not recorded | 6a1e6438dbd7150506f85881 | | Lie Diffuser Actor | 3.368 | Not recorded | 6a1e6438dbd7150506f85881 | | 3D Diffuser Actor | 3.288 | Not recorded | 6a1e6438dbd7150506f85881 | | 3D Diffuser Actor | 3.27 | Not recorded | 6a1e6438dbd7150506f85881 | | OpenVLA-OFT | 3.27 | Not recorded | 6a21067bc10484bac23efff5 | | Lie Diffuser Actor | 3.239 | Not recorded | 6a1e6438dbd7150506f85881 | | GR-1 | 3.06 | Not recorded | 68fa7c0b1e209131cea8c0d9 | ### X-VLA Average Sequence Length: 4.43. Result ID: 6a2123b6fbc373592c1c25b2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Suite":"CALVIN ABC→D"}. This record does not assert independent reproduction. ### VLA-Adapter Average Sequence Length: 4.42. Result ID: 6a2123b6fbc373592c1c25b1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Suite":"CALVIN ABC→D"}. This record does not assert independent reproduction. ### UniJEPA Average Sequence Length: 4.11. Result ID: 6a056f7b47f34ce924ebc9fd. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Visual prediction":"true","Input resolution":"224x224","Input view":"single third-view"}. This record does not assert independent reproduction. ### UP-VLA\* Average Sequence Length: 4.08. Result ID: 6a056f7b47f34ce924ebc9fc. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Visual prediction":"true","Input resolution":"224x224","Input view":"single third-view","Reproduction status":"reproduced"}. This record does not assert independent reproduction. ### S2 Average Sequence Length: 3.95. Result ID: 6a2123b6fbc373592c1c25b4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Suite":"CALVIN ABC→D","interface":"Specify More language interface"}. This record does not assert independent reproduction. ### π0.5 Average Sequence Length: 3.92. Result ID: 6a2123b6fbc373592c1c25b3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Suite":"CALVIN ABC→D"}. This record does not assert independent reproduction. ### UniVLA Average Sequence Length: 3.8. Result ID: 6a2123b6fbc373592c1c25b0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Suite":"CALVIN ABC→D"}. This record does not assert independent reproduction. ### Lie Diffuser Actor Average Sequence Length: 3.584. Result ID: 6a1eca2afbc373592c1a68ca. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training Iterations":"300K"}. This record does not assert independent reproduction. ### VPP\* Average Sequence Length: 3.58. Result ID: 6a056f7b47f34ce924ebc9fb. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Visual prediction":"true","Input resolution":"224x224","Input view":"single third-view","Reproduction status":"reproduced"}. This record does not assert independent reproduction. ### Lie Diffuser Actor Average Sequence Length: 3.512. Result ID: 6a1eca2afbc373592c1a68c6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training Iterations":"300K"}. This record does not assert independent reproduction. ### Lie Diffuser Actor Average Sequence Length: 3.474. Result ID: 6a1eca2afbc373592c1a68c5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training Iterations":"300K","Lie Space Diffusion":"excluded"}. This record does not assert independent reproduction. ### Lie Diffuser Actor Average Sequence Length: 3.404. Result ID: 6a1eca2afbc373592c1a68c8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training Iterations":"300K","GAT Encoder":"excluded"}. This record does not assert independent reproduction. ### Lie Diffuser Actor Average Sequence Length: 3.368. Result ID: 6a1eca2afbc373592c1a68c4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training Iterations":"600K","GAT Encoder":"excluded"}. This record does not assert independent reproduction. ### 3D Diffuser Actor Average Sequence Length: 3.288. Result ID: 6a1eca2afbc373592c1a68c7. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Iterations":"800K"}. This record does not assert independent reproduction. ### 3D Diffuser Actor Average Sequence Length: 3.27. Result ID: 6a1eca2afbc373592c1a68c3. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Iterations":"600K"}. This record does not assert independent reproduction. ### OpenVLA-OFT Average Sequence Length: 3.27. Result ID: 6a2123b6fbc373592c1c25af. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Suite":"CALVIN ABC→D"}. This record does not assert independent reproduction. ### Lie Diffuser Actor Average Sequence Length: 3.239. Result ID: 6a1eca2afbc373592c1a68c9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training Iterations":"400K","Lie Space Diffusion":"excluded"}. This record does not assert independent reproduction. ### GR-1 Average Sequence Length: 3.06. Result ID: 6a056f7b47f34ce924ebc9fa. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Visual prediction":"true","Input resolution":"224x224","Input view":"single third-view"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.