# Long-horizon language-conditioned policy learning on CALVIN Source: [Long-horizon language-conditioned policy learning on CALVIN]() Collection index: [llms.txt]() Data updated: 2026-03-25T14:00:12.346Z Evaluation of long-horizon policy learning conditioned on natural language instructions using the CALVIN dataset. Performance is typically measured by sequential task completion rates (1/5 to 5/5 steps) and average trajectory metrics. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Long-horizon language-conditioned policy learning | | Dataset (including any stated split) | CALVIN | | Metric | Success Rate (Step 5/5) | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 16 of 16 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Success Rate (Step 5/5)](): Higher is better; unit: PERCENT. - [Success Rate (Step 4/5)](): Higher is better; unit: PERCENT. - [Success Rate (Step 3/5)](): Higher is better; unit: PERCENT. - [Success Rate (Step 2/5)](): Higher is better; unit: PERCENT. - [Success Rate (Step 1/5)](): Higher is better; unit: PERCENT. - [Average Score](): Higher is better; unit: SCALAR. ## Recorded results | Method | Success Rate (Step 5/5) | Result date | Report paper | | --- | --- | --- | --- | | VLA-IAP | 98.4 | Not recorded | 69c36c430636e5bf4afc66bb | | VLA-IAP | 98.4 | Not recorded | 69c36c430636e5bf4afc66bb | | DivPrune | 98.3 | Not recorded | 69c36c430636e5bf4afc66bb | | VLA-Cache | 98.2 | Not recorded | 69c36c430636e5bf4afc66bb | | VLA-Cache | 98.1 | Not recorded | 69c36c430636e5bf4afc66bb | | DivPrune | 98 | Not recorded | 69c36c430636e5bf4afc66bb | | SparseVLM | 97.9 | Not recorded | 69c36c430636e5bf4afc66bb | | Vanilla | 97.8 | Not recorded | 69c36c430636e5bf4afc66bb | | VLA-IAP | 97.8 | Not recorded | 69c36c430636e5bf4afc66bb | | FastV | 97.6 | Not recorded | 69c36c430636e5bf4afc66bb | | SparseVLM | 97.4 | Not recorded | 69c36c430636e5bf4afc66bb | | VLA-Cache | 97.1 | Not recorded | 69c36c430636e5bf4afc66bb | | DivPrune | 96.8 | Not recorded | 69c36c430636e5bf4afc66bb | | SparseVLM | 96.3 | Not recorded | 69c36c430636e5bf4afc66bb | | FastV | 94.2 | Not recorded | 69c36c430636e5bf4afc66bb | | FastV | 89.8 | Not recorded | 69c36c430636e5bf4afc66bb | ### VLA-IAP Success Rate (Step 5/5): 98.4. Result ID: 69c3ea6747f34ce924ad9b7b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Token Retention Ratio":"70%","Pruning Ratio":"30%"}. This record does not assert independent reproduction. ### VLA-IAP Success Rate (Step 5/5): 98.4. Result ID: 69c3ea6747f34ce924ad9b80. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Token Retention Ratio":"50%","Pruning Ratio":"50%"}. This record does not assert independent reproduction. ### DivPrune Success Rate (Step 5/5): 98.3. Result ID: 69c3ea6747f34ce924ad9b7e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Token Retention Ratio":"50%","Pruning Ratio":"50%"}. This record does not assert independent reproduction. ### VLA-Cache Success Rate (Step 5/5): 98.2. Result ID: 69c3ea6747f34ce924ad9b7a. [Reporting paper]() Recorded evaluation configuration: {"Token Retention Ratio":"70%","Pruning Ratio":"30%"}. This record does not assert independent reproduction. ### VLA-Cache Success Rate (Step 5/5): 98.1. Result ID: 69c3ea6747f34ce924ad9b7f. [Reporting paper]() Recorded evaluation configuration: {"Token Retention Ratio":"50%","Pruning Ratio":"50%"}. This record does not assert independent reproduction. ### DivPrune Success Rate (Step 5/5): 98. Result ID: 69c3ea6747f34ce924ad9b79. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Token Retention Ratio":"70%","Pruning Ratio":"30%"}. This record does not assert independent reproduction. ### SparseVLM Success Rate (Step 5/5): 97.9. Result ID: 69c3ea6747f34ce924ad9b7d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Token Retention Ratio":"50%","Pruning Ratio":"50%"}. This record does not assert independent reproduction. ### Vanilla Success Rate (Step 5/5): 97.8. Result ID: 69c3ea6747f34ce924ad9b76. [Reporting paper]() Recorded evaluation configuration: {"Base Model":"Vanilla","Token Retention Ratio":"100%"}. This record does not assert independent reproduction. ### VLA-IAP Success Rate (Step 5/5): 97.8. Result ID: 69c3ea6747f34ce924ad9b85. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Token Retention Ratio":"30%","Pruning Ratio":"70%"}. This record does not assert independent reproduction. ### FastV Success Rate (Step 5/5): 97.6. Result ID: 69c3ea6747f34ce924ad9b77. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Token Retention Ratio":"70%","Pruning Ratio":"30%"}. This record does not assert independent reproduction. ### SparseVLM Success Rate (Step 5/5): 97.4. Result ID: 69c3ea6747f34ce924ad9b78. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Token Retention Ratio":"70%","Pruning Ratio":"30%"}. This record does not assert independent reproduction. ### VLA-Cache Success Rate (Step 5/5): 97.1. Result ID: 69c3ea6747f34ce924ad9b84. [Reporting paper]() Recorded evaluation configuration: {"Token Retention Ratio":"30%","Pruning Ratio":"70%"}. This record does not assert independent reproduction. ### DivPrune Success Rate (Step 5/5): 96.8. Result ID: 69c3ea6747f34ce924ad9b83. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Token Retention Ratio":"30%","Pruning Ratio":"70%"}. This record does not assert independent reproduction. ### SparseVLM Success Rate (Step 5/5): 96.3. Result ID: 69c3ea6747f34ce924ad9b82. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Token Retention Ratio":"30%","Pruning Ratio":"70%"}. This record does not assert independent reproduction. ### FastV Success Rate (Step 5/5): 94.2. Result ID: 69c3ea6747f34ce924ad9b7c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Token Retention Ratio":"50%","Pruning Ratio":"50%"}. This record does not assert independent reproduction. ### FastV Success Rate (Step 5/5): 89.8. Result ID: 69c3ea6747f34ce924ad9b81. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Token Retention Ratio":"30%","Pruning Ratio":"70%"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.