# Failure Detection on LIBERO-10 Seen Tasks Source: [Failure Detection on LIBERO-10 Seen Tasks]() Collection index: [llms.txt]() Data updated: 2026-06-01T08:18:13.447Z Evaluation of Failure Detection performance on the LIBERO-10 Seen Tasks dataset, measured using bACC, wACC, and TWA. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Failure Detection | | Dataset (including any stated split) | LIBERO-10 Seen Tasks | | Metric | bACC | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 28 of 28 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [bACC](): Higher is better; unit: PERCENT. - [wACC](): Higher is better; unit: PERCENT. - [TWA](): Higher is better; unit: PERCENT. ## Recorded results | Method | bACC | Result date | Report paper | | --- | --- | --- | --- | | Hide-and-Seek | 88.5 | Not recorded | 6a1d11dbdbd7150506f83d90 | | SAFE-MLP | 87 | Not recorded | 6a1d11dbdbd7150506f83d90 | | Hide-and-Seek | 85.2 | Not recorded | 6a1d11dbdbd7150506f83d90 | | SAFE-MLP | 82.3 | Not recorded | 6a1d11dbdbd7150506f83d90 | | Mahalanobis | 78.8 | Not recorded | 6a1d11dbdbd7150506f83d90 | | LogpZO | 78.6 | Not recorded | 6a1d11dbdbd7150506f83d90 | | SAFE-LSTM | 77.4 | Not recorded | 6a1d11dbdbd7150506f83d90 | | Cosine k-NN | 75.8 | Not recorded | 6a1d11dbdbd7150506f83d90 | | RND | 70.6 | Not recorded | 6a1d11dbdbd7150506f83d90 | | EigenScore | 69.1 | Not recorded | 6a1d11dbdbd7150506f83d90 | | ACE | 67.2 | Not recorded | 6a1d11dbdbd7150506f83d90 | | Mahalanobis | 67 | Not recorded | 6a1d11dbdbd7150506f83d90 | | SAFE-LSTM | 67 | Not recorded | 6a1d11dbdbd7150506f83d90 | | STAC | 66.5 | Not recorded | 6a1d11dbdbd7150506f83d90 | | EigenScore | 66.3 | Not recorded | 6a1d11dbdbd7150506f83d90 | | LogpZO | 65.8 | Not recorded | 6a1d11dbdbd7150506f83d90 | | STAC | 61.9 | Not recorded | 6a1d11dbdbd7150506f83d90 | | ACE | 60.7 | Not recorded | 6a1d11dbdbd7150506f83d90 | | PCA KMeans | 59.1 | Not recorded | 6a1d11dbdbd7150506f83d90 | | Cosine k-NN | 58 | Not recorded | 6a1d11dbdbd7150506f83d90 | | Cluster Entropy | 55.2 | Not recorded | 6a1d11dbdbd7150506f83d90 | | Cluster Entropy | 54.8 | Not recorded | 6a1d11dbdbd7150506f83d90 | | PCA KMeans | 53.4 | Not recorded | 6a1d11dbdbd7150506f83d90 | | RND | 52.6 | Not recorded | 6a1d11dbdbd7150506f83d90 | | Max Entropy | 51.4 | Not recorded | 6a1d11dbdbd7150506f83d90 | | Avg. Entropy | 51.3 | Not recorded | 6a1d11dbdbd7150506f83d90 | | Avg. NLL | 50.6 | Not recorded | 6a1d11dbdbd7150506f83d90 | | Max NLL | 49.4 | Not recorded | 6a1d11dbdbd7150506f83d90 | ### Hide-and-Seek bACC: 88.5. Result ID: 6a1d4034fbc373592c165d63. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy":"π0","Category":"Class."}. This record does not assert independent reproduction. ### SAFE-MLP bACC: 87. Result ID: 6a1d4034fbc373592c165d62. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy":"π0","Category":"Class.","Venue":"NeurIPS'25"}. This record does not assert independent reproduction. ### Hide-and-Seek bACC: 85.2. Result ID: 6a1d4034fbc373592c165d57. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy":"OpenVLA","Category":"Class."}. This record does not assert independent reproduction. ### SAFE-MLP bACC: 82.3. Result ID: 6a1d4034fbc373592c165d56. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy":"OpenVLA","Category":"Class.","Venue":"NeurIPS'25"}. This record does not assert independent reproduction. ### Mahalanobis bACC: 78.8. Result ID: 6a1d4034fbc373592c165d58. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Policy":"π0","Category":"OOD Det.","Venue":"ICML'18"}. This record does not assert independent reproduction. ### LogpZO bACC: 78.6. Result ID: 6a1d4034fbc373592c165d5c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy":"π0","Category":"OOD Det.","Venue":"RSS'25"}. This record does not assert independent reproduction. ### SAFE-LSTM bACC: 77.4. Result ID: 6a1d4034fbc373592c165d61. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy":"π0","Category":"Class.","Venue":"NeurIPS'25"}. This record does not assert independent reproduction. ### Cosine k-NN bACC: 75.8. Result ID: 6a1d4034fbc373592c165d4d. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Policy":"OpenVLA","Category":"OOD Det.","Venue":"ICML'22"}. This record does not assert independent reproduction. ### RND bACC: 70.6. Result ID: 6a1d4034fbc373592c165d5a. [Reporting paper]() Recorded evaluation configuration: {"Policy":"π0","Category":"OOD Det.","Venue":"ICML'24"}. This record does not assert independent reproduction. ### EigenScore bACC: 69.1. Result ID: 6a1d4034fbc373592c165d5e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy":"π0","Category":"Multi.","Venue":"ICLR'24"}. This record does not assert independent reproduction. ### ACE bACC: 67.2. Result ID: 6a1d4034fbc373592c165d60. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy":"π0","Category":"Multi.","Venue":"NeurIPS'25"}. This record does not assert independent reproduction. ### Mahalanobis bACC: 67. Result ID: 6a1d4034fbc373592c165d4c. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Policy":"OpenVLA","Category":"OOD Det.","Venue":"ICML'18"}. This record does not assert independent reproduction. ### SAFE-LSTM bACC: 67. Result ID: 6a1d4034fbc373592c165d55. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy":"OpenVLA","Category":"Class.","Venue":"NeurIPS'25"}. This record does not assert independent reproduction. ### STAC bACC: 66.5. Result ID: 6a1d4034fbc373592c165d53. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy":"OpenVLA","Category":"Multi.","Venue":"CoRL'24","note":"Adapted to single-step output"}. This record does not assert independent reproduction. ### EigenScore bACC: 66.3. Result ID: 6a1d4034fbc373592c165d52. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy":"OpenVLA","Category":"Multi.","Venue":"ICLR'24"}. This record does not assert independent reproduction. ### LogpZO bACC: 65.8. Result ID: 6a1d4034fbc373592c165d50. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy":"OpenVLA","Category":"OOD Det.","Venue":"RSS'25"}. This record does not assert independent reproduction. ### STAC bACC: 61.9. Result ID: 6a1d4034fbc373592c165d5f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy":"π0","Category":"Multi.","Venue":"CoRL'24"}. This record does not assert independent reproduction. ### ACE bACC: 60.7. Result ID: 6a1d4034fbc373592c165d54. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy":"OpenVLA","Category":"Multi.","Venue":"NeurIPS'25"}. This record does not assert independent reproduction. ### PCA KMeans bACC: 59.1. Result ID: 6a1d4034fbc373592c165d5b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy":"π0","Category":"OOD Det.","Venue":"CoRL'24"}. This record does not assert independent reproduction. ### Cosine k-NN bACC: 58. Result ID: 6a1d4034fbc373592c165d59. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Policy":"π0","Category":"OOD Det.","Venue":"ICML'22"}. This record does not assert independent reproduction. ### Cluster Entropy bACC: 55.2. Result ID: 6a1d4034fbc373592c165d51. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy":"OpenVLA","Category":"Multi.","Venue":"ICLR'23"}. This record does not assert independent reproduction. ### Cluster Entropy bACC: 54.8. Result ID: 6a1d4034fbc373592c165d5d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy":"π0","Category":"Multi.","Venue":"ICLR'23"}. This record does not assert independent reproduction. ### PCA KMeans bACC: 53.4. Result ID: 6a1d4034fbc373592c165d4e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy":"OpenVLA","Category":"OOD Det.","Venue":"CoRL'24"}. This record does not assert independent reproduction. ### RND bACC: 52.6. Result ID: 6a1d4034fbc373592c165d4f. [Reporting paper]() Recorded evaluation configuration: {"Policy":"OpenVLA","Category":"OOD Det.","Venue":"ICML'24"}. This record does not assert independent reproduction. ### Max Entropy bACC: 51.4. Result ID: 6a1d4034fbc373592c165d48. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy":"OpenVLA","Category":"Token Unc.","Venue":"ICLR'21"}. This record does not assert independent reproduction. ### Avg. Entropy bACC: 51.3. Result ID: 6a1d4034fbc373592c165d49. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy":"OpenVLA","Category":"Token Unc.","Venue":"ICLR'21"}. This record does not assert independent reproduction. ### Avg. NLL bACC: 50.6. Result ID: 6a1d4034fbc373592c165d4b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy":"OpenVLA","Category":"Token Unc.","Venue":"ICLR'23"}. This record does not assert independent reproduction. ### Max NLL bACC: 49.4. Result ID: 6a1d4034fbc373592c165d4a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Policy":"OpenVLA","Category":"Token Unc.","Venue":"ICLR'23"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.