# Failure Detection on LIBERO 10 Unseen Tasks Source: [Failure Detection on LIBERO 10 Unseen Tasks]() Collection index: [llms.txt]() Data updated: 2026-06-01T08:18:13.476Z Evaluation of Failure Detection performance on the LIBERO 10 Unseen Tasks dataset, measured using bACC, wACC, and TWA. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Failure Detection | | Dataset (including any stated split) | LIBERO 10 Unseen Tasks | | Metric | bACC | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 28 of 28 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [bACC](): Higher is better; unit: PERCENT. - [wACC](): Higher is better; unit: PERCENT. - [TWA](): Higher is better; unit: PERCENT. ## Recorded results | Method | bACC | Result date | Report paper | | --- | --- | --- | --- | | Hide-and-Seek | 89.2 | Not recorded | 6a1d11dbdbd7150506f83d90 | | Hide-and-Seek | 83.4 | Not recorded | 6a1d11dbdbd7150506f83d90 | | SAFE-MLP | 80.1 | Not recorded | 6a1d11dbdbd7150506f83d90 | | SAFE-MLP | 77.5 | Not recorded | 6a1d11dbdbd7150506f83d90 | | Cluster Entropy | 73.2 | Not recorded | 6a1d11dbdbd7150506f83d90 | | SAFE-LSTM | 68.6 | Not recorded | 6a1d11dbdbd7150506f83d90 | | LogpZO | 68.1 | Not recorded | 6a1d11dbdbd7150506f83d90 | | EigenScore | 67.1 | Not recorded | 6a1d11dbdbd7150506f83d90 | | Cosine k-NN | 63.9 | Not recorded | 6a1d11dbdbd7150506f83d90 | | Mahalanobis | 63.4 | Not recorded | 6a1d11dbdbd7150506f83d90 | | STAC | 62.4 | Not recorded | 6a1d11dbdbd7150506f83d90 | | Cosine k-NN | 62.3 | Not recorded | 6a1d11dbdbd7150506f83d90 | | EigenScore | 61.3 | Not recorded | 6a1d11dbdbd7150506f83d90 | | ACE | 60.9 | Not recorded | 6a1d11dbdbd7150506f83d90 | | SAFE-LSTM | 60.9 | Not recorded | 6a1d11dbdbd7150506f83d90 | | ACE | 60.4 | Not recorded | 6a1d11dbdbd7150506f83d90 | | Cluster Entropy | 57 | Not recorded | 6a1d11dbdbd7150506f83d90 | | LogpZO | 54.5 | Not recorded | 6a1d11dbdbd7150506f83d90 | | Max NLL | 52.5 | Not recorded | 6a1d11dbdbd7150506f83d90 | | Max Entropy | 52.1 | Not recorded | 6a1d11dbdbd7150506f83d90 | | RND | 52 | Not recorded | 6a1d11dbdbd7150506f83d90 | | Mahalanobis | 51.3 | Not recorded | 6a1d11dbdbd7150506f83d90 | | RND | 50.9 | Not recorded | 6a1d11dbdbd7150506f83d90 | | Avg. NLL | 49.5 | Not recorded | 6a1d11dbdbd7150506f83d90 | | Avg. Entropy | 49.1 | Not recorded | 6a1d11dbdbd7150506f83d90 | | STAC | 49 | Not recorded | 6a1d11dbdbd7150506f83d90 | | PCA KMeans | 48.6 | Not recorded | 6a1d11dbdbd7150506f83d90 | | PCA KMeans | 46.1 | Not recorded | 6a1d11dbdbd7150506f83d90 | ### Hide-and-Seek bACC: 89.2. Result ID: 6a1d4034fbc373592c165d7f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy":"π0","Category":"Class."}. This record does not assert independent reproduction. ### Hide-and-Seek bACC: 83.4. Result ID: 6a1d4034fbc373592c165d73. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy":"OpenVLA","Category":"Class."}. This record does not assert independent reproduction. ### SAFE-MLP bACC: 80.1. Result ID: 6a1d4034fbc373592c165d7e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy":"π0","Category":"Class.","Venue":"NeurIPS'25"}. This record does not assert independent reproduction. ### SAFE-MLP bACC: 77.5. Result ID: 6a1d4034fbc373592c165d72. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy":"OpenVLA","Category":"Class.","Venue":"NeurIPS'25"}. This record does not assert independent reproduction. ### Cluster Entropy bACC: 73.2. Result ID: 6a1d4034fbc373592c165d79. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy":"π0","Category":"Multi.","Venue":"ICLR'23"}. This record does not assert independent reproduction. ### SAFE-LSTM bACC: 68.6. Result ID: 6a1d4034fbc373592c165d71. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy":"OpenVLA","Category":"Class.","Venue":"NeurIPS'25"}. This record does not assert independent reproduction. ### LogpZO bACC: 68.1. Result ID: 6a1d4034fbc373592c165d78. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy":"π0","Category":"OOD Det.","Venue":"RSS'25"}. This record does not assert independent reproduction. ### EigenScore bACC: 67.1. Result ID: 6a1d4034fbc373592c165d6e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy":"OpenVLA","Category":"Multi.","Venue":"ICLR'24"}. This record does not assert independent reproduction. ### Cosine k-NN bACC: 63.9. Result ID: 6a1d4034fbc373592c165d69. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Policy":"OpenVLA","Category":"OOD Det.","Venue":"ICML'22"}. This record does not assert independent reproduction. ### Mahalanobis bACC: 63.4. Result ID: 6a1d4034fbc373592c165d74. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Policy":"π0","Category":"OOD Det.","Venue":"ICML'18"}. This record does not assert independent reproduction. ### STAC bACC: 62.4. Result ID: 6a1d4034fbc373592c165d6f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy":"OpenVLA","Category":"Multi.","Venue":"CoRL'24"}. This record does not assert independent reproduction. ### Cosine k-NN bACC: 62.3. Result ID: 6a1d4034fbc373592c165d75. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Policy":"π0","Category":"OOD Det.","Venue":"ICML'22"}. This record does not assert independent reproduction. ### EigenScore bACC: 61.3. Result ID: 6a1d4034fbc373592c165d7a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy":"π0","Category":"Multi.","Venue":"ICLR'24"}. This record does not assert independent reproduction. ### ACE bACC: 60.9. Result ID: 6a1d4034fbc373592c165d7c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy":"π0","Category":"Multi.","Venue":"NeurIPS'25"}. This record does not assert independent reproduction. ### SAFE-LSTM bACC: 60.9. Result ID: 6a1d4034fbc373592c165d7d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy":"π0","Category":"Class.","Venue":"NeurIPS'25"}. This record does not assert independent reproduction. ### ACE bACC: 60.4. Result ID: 6a1d4034fbc373592c165d70. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy":"OpenVLA","Category":"Multi.","Venue":"NeurIPS'25"}. This record does not assert independent reproduction. ### Cluster Entropy bACC: 57. Result ID: 6a1d4034fbc373592c165d6d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy":"OpenVLA","Category":"Multi.","Venue":"ICLR'23"}. This record does not assert independent reproduction. ### LogpZO bACC: 54.5. Result ID: 6a1d4034fbc373592c165d6c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy":"OpenVLA","Category":"OOD Det.","Venue":"RSS'25"}. This record does not assert independent reproduction. ### Max NLL bACC: 52.5. Result ID: 6a1d4034fbc373592c165d66. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy":"OpenVLA","Category":"Token Unc.","Venue":"ICLR'23"}. This record does not assert independent reproduction. ### Max Entropy bACC: 52.1. Result ID: 6a1d4034fbc373592c165d64. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy":"OpenVLA","Category":"Token Unc.","Venue":"ICLR'21"}. This record does not assert independent reproduction. ### RND bACC: 52. Result ID: 6a1d4034fbc373592c165d76. [Reporting paper]() Recorded evaluation configuration: {"Policy":"π0","Category":"OOD Det.","Venue":"ICML'24"}. This record does not assert independent reproduction. ### Mahalanobis bACC: 51.3. Result ID: 6a1d4034fbc373592c165d68. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Policy":"OpenVLA","Category":"OOD Det.","Venue":"ICML'18"}. This record does not assert independent reproduction. ### RND bACC: 50.9. Result ID: 6a1d4034fbc373592c165d6b. [Reporting paper]() Recorded evaluation configuration: {"Policy":"OpenVLA","Category":"OOD Det.","Venue":"ICML'24"}. This record does not assert independent reproduction. ### Avg. NLL bACC: 49.5. Result ID: 6a1d4034fbc373592c165d67. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy":"OpenVLA","Category":"Token Unc.","Venue":"ICLR'23"}. This record does not assert independent reproduction. ### Avg. Entropy bACC: 49.1. Result ID: 6a1d4034fbc373592c165d65. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy":"OpenVLA","Category":"Token Unc.","Venue":"ICLR'21"}. This record does not assert independent reproduction. ### STAC bACC: 49. Result ID: 6a1d4034fbc373592c165d7b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy":"π0","Category":"Multi.","Venue":"CoRL'24"}. This record does not assert independent reproduction. ### PCA KMeans bACC: 48.6. Result ID: 6a1d4034fbc373592c165d6a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy":"OpenVLA","Category":"OOD Det.","Venue":"CoRL'24"}. This record does not assert independent reproduction. ### PCA KMeans bACC: 46.1. Result ID: 6a1d4034fbc373592c165d77. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy":"π0","Category":"OOD Det.","Venue":"CoRL'24"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.