# Failure Detection on LIBERO unseen Source: [Failure Detection on LIBERO unseen]() Collection index: [llms.txt]() Data updated: 2026-04-23T10:49:52.061Z Evaluation of Failure Detection performance on the LIBERO unseen dataset, measured using the Brier Score. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Failure Detection | | Dataset (including any stated split) | LIBERO unseen | | Metric | Brier Score | | Unit | SCALAR | | Direction | Lower is better | | Results loaded | 37 of 37 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Brier Score](): Lower is better; unit: SCALAR. ## Recorded results | Method | Brier Score | Result date | Report paper | | --- | --- | --- | --- | | SAFE-RNN-TDQC (Ours) | 0.097 | Not recorded | 69e9a78555a3c32af89f6fea | | SAFE-RNN-TDQC (Ours) | 0.1 | Not recorded | 69e9a78555a3c32af89f6fea | | RNN-TDQC (Ours) | 0.107 | Not recorded | 69e9a78555a3c32af89f6fea | | SAFE-MLP-TDQC (Ours) | 0.128 | Not recorded | 69e9a78555a3c32af89f6fea | | SAFE-MLP-TDQC (Ours) | 0.131 | Not recorded | 69e9a78555a3c32af89f6fea | | SAFE-MLP BCE | 0.137 | Not recorded | 69e9a78555a3c32af89f6fea | | RNN-TDQC (Ours) | 0.141 | Not recorded | 69e9a78555a3c32af89f6fea | | SAFE-RNN | 0.148 | Not recorded | 69e9a78555a3c32af89f6fea | | SAFE-MLP-TDQC (Ours) | 0.15 | Not recorded | 69e9a78555a3c32af89f6fea | | RNN-BCE | 0.152 | Not recorded | 69e9a78555a3c32af89f6fea | | SAFE-MLP BCE | 0.158 | Not recorded | 69e9a78555a3c32af89f6fea | | RNN-BCE | 0.158 | Not recorded | 69e9a78555a3c32af89f6fea | | SAFE-RNN | 0.162 | Not recorded | 69e9a78555a3c32af89f6fea | | SAFE-MLP BCE | 0.162 | Not recorded | 69e9a78555a3c32af89f6fea | | SAFE-RNN-TDQC (Ours) | 0.163 | Not recorded | 69e9a78555a3c32af89f6fea | | SAFE-RNN | 0.172 | Not recorded | 69e9a78555a3c32af89f6fea | | RNN-TDQC (Ours) | 0.197 | Not recorded | 69e9a78555a3c32af89f6fea | | RNN-BCE | 0.206 | Not recorded | 69e9a78555a3c32af89f6fea | | SAFE-RNN-TDQC (Ours) | 0.218 | Not recorded | 69e9a78555a3c32af89f6fea | | Avg prob. | 0.218 | Not recorded | 69e9a78555a3c32af89f6fea | | Avg entropy | 0.222 | Not recorded | 69e9a78555a3c32af89f6fea | | SAFE-MLP-TDQC (Ours) | 0.229 | Not recorded | 69e9a78555a3c32af89f6fea | | SAFE-MLP BCE | 0.231 | Not recorded | 69e9a78555a3c32af89f6fea | | Running Avg prob. | 0.238 | Not recorded | 69e9a78555a3c32af89f6fea | | SAFE-RNN | 0.255 | Not recorded | 69e9a78555a3c32af89f6fea | | Running Avg entropy | 0.264 | Not recorded | 69e9a78555a3c32af89f6fea | | Running Avg entropy | 0.273 | Not recorded | 69e9a78555a3c32af89f6fea | | Avg entropy | 0.313 | Not recorded | 69e9a78555a3c32af89f6fea | | Max prob. | 0.318 | Not recorded | 69e9a78555a3c32af89f6fea | | Running Avg entropy | 0.33 | Not recorded | 69e9a78555a3c32af89f6fea | | Running Avg prob. | 0.356 | Not recorded | 69e9a78555a3c32af89f6fea | | Avg prob. | 0.364 | Not recorded | 69e9a78555a3c32af89f6fea | | Max prob. | 0.39 | Not recorded | 69e9a78555a3c32af89f6fea | | Avg entropy | 0.391 | Not recorded | 69e9a78555a3c32af89f6fea | | Avg prob. | 0.532 | Not recorded | 69e9a78555a3c32af89f6fea | | Running Avg prob. | 0.544 | Not recorded | 69e9a78555a3c32af89f6fea | | Max prob. | 0.899 | Not recorded | 69e9a78555a3c32af89f6fea | ### SAFE-RNN-TDQC (Ours) Brier Score: 0.097. Result ID: 69e9f93547f34ce924d1f465. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"VLA Model":"π0"}. This record does not assert independent reproduction. ### SAFE-RNN-TDQC (Ours) Brier Score: 0.1. Result ID: 69e9f93547f34ce924d1f454. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"VLA Model":"UniVLA"}. This record does not assert independent reproduction. ### RNN-TDQC (Ours) Brier Score: 0.107. Result ID: 69e9f93547f34ce924d1f458. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"VLA Model":"UniVLA"}. This record does not assert independent reproduction. ### SAFE-MLP-TDQC (Ours) Brier Score: 0.128. Result ID: 69e9f93547f34ce924d1f467. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"VLA Model":"π0"}. This record does not assert independent reproduction. ### SAFE-MLP-TDQC (Ours) Brier Score: 0.131. Result ID: 69e9f93547f34ce924d1f456. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"VLA Model":"UniVLA"}. This record does not assert independent reproduction. ### SAFE-MLP BCE Brier Score: 0.137. Result ID: 69e9f93547f34ce924d1f466. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"VLA Model":"π0"}. This record does not assert independent reproduction. ### RNN-TDQC (Ours) Brier Score: 0.141. Result ID: 69e9f93547f34ce924d1f463. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"VLA Model":"π0-FAST"}. This record does not assert independent reproduction. ### SAFE-RNN Brier Score: 0.148. Result ID: 69e9f93547f34ce924d1f45e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"VLA Model":"π0-FAST"}. This record does not assert independent reproduction. ### SAFE-MLP-TDQC (Ours) Brier Score: 0.15. Result ID: 69e9f93547f34ce924d1f461. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"VLA Model":"π0-FAST"}. This record does not assert independent reproduction. ### RNN-BCE Brier Score: 0.152. Result ID: 69e9f93547f34ce924d1f457. [Reporting paper]() Recorded evaluation configuration: {"VLA Model":"UniVLA"}. This record does not assert independent reproduction. ### SAFE-MLP BCE Brier Score: 0.158. Result ID: 69e9f93547f34ce924d1f455. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"VLA Model":"UniVLA"}. This record does not assert independent reproduction. ### RNN-BCE Brier Score: 0.158. Result ID: 69e9f93547f34ce924d1f462. [Reporting paper]() Recorded evaluation configuration: {"VLA Model":"π0-FAST"}. This record does not assert independent reproduction. ### SAFE-RNN Brier Score: 0.162. Result ID: 69e9f93547f34ce924d1f453. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"VLA Model":"UniVLA"}. This record does not assert independent reproduction. ### SAFE-MLP BCE Brier Score: 0.162. Result ID: 69e9f93547f34ce924d1f460. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"VLA Model":"π0-FAST"}. This record does not assert independent reproduction. ### SAFE-RNN-TDQC (Ours) Brier Score: 0.163. Result ID: 69e9f93547f34ce924d1f45f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"VLA Model":"π0-FAST"}. This record does not assert independent reproduction. ### SAFE-RNN Brier Score: 0.172. Result ID: 69e9f93547f34ce924d1f464. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"VLA Model":"π0"}. This record does not assert independent reproduction. ### RNN-TDQC (Ours) Brier Score: 0.197. Result ID: 69e9f93547f34ce924d1f44d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"VLA Model":"OpenVLA"}. This record does not assert independent reproduction. ### RNN-BCE Brier Score: 0.206. Result ID: 69e9f93547f34ce924d1f44c. [Reporting paper]() Recorded evaluation configuration: {"VLA Model":"OpenVLA"}. This record does not assert independent reproduction. ### SAFE-RNN-TDQC (Ours) Brier Score: 0.218. Result ID: 69e9f93547f34ce924d1f449. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"VLA Model":"OpenVLA"}. This record does not assert independent reproduction. ### Avg prob. Brier Score: 0.218. Result ID: 69e9f93547f34ce924d1f45a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"VLA Model":"π0-FAST"}. This record does not assert independent reproduction. ### Avg entropy Brier Score: 0.222. Result ID: 69e9f93547f34ce924d1f45c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"VLA Model":"π0-FAST"}. This record does not assert independent reproduction. ### SAFE-MLP-TDQC (Ours) Brier Score: 0.229. Result ID: 69e9f93547f34ce924d1f44b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"VLA Model":"OpenVLA"}. This record does not assert independent reproduction. ### SAFE-MLP BCE Brier Score: 0.231. Result ID: 69e9f93547f34ce924d1f44a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"VLA Model":"OpenVLA"}. This record does not assert independent reproduction. ### Running Avg prob. Brier Score: 0.238. Result ID: 69e9f93547f34ce924d1f45b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"VLA Model":"π0-FAST"}. This record does not assert independent reproduction. ### SAFE-RNN Brier Score: 0.255. Result ID: 69e9f93547f34ce924d1f448. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"VLA Model":"OpenVLA"}. This record does not assert independent reproduction. ### Running Avg entropy Brier Score: 0.264. Result ID: 69e9f93547f34ce924d1f45d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"VLA Model":"π0-FAST"}. This record does not assert independent reproduction. ### Running Avg entropy Brier Score: 0.273. Result ID: 69e9f93547f34ce924d1f447. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"VLA Model":"OpenVLA"}. This record does not assert independent reproduction. ### Avg entropy Brier Score: 0.313. Result ID: 69e9f93547f34ce924d1f446. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"VLA Model":"OpenVLA"}. This record does not assert independent reproduction. ### Max prob. Brier Score: 0.318. Result ID: 69e9f93547f34ce924d1f459. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"VLA Model":"π0-FAST"}. This record does not assert independent reproduction. ### Running Avg entropy Brier Score: 0.33. Result ID: 69e9f93547f34ce924d1f452. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"VLA Model":"UniVLA"}. This record does not assert independent reproduction. ### Running Avg prob. Brier Score: 0.356. Result ID: 69e9f93547f34ce924d1f445. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"VLA Model":"OpenVLA"}. This record does not assert independent reproduction. ### Avg prob. Brier Score: 0.364. Result ID: 69e9f93547f34ce924d1f444. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"VLA Model":"OpenVLA"}. This record does not assert independent reproduction. ### Max prob. Brier Score: 0.39. Result ID: 69e9f93547f34ce924d1f443. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"VLA Model":"OpenVLA"}. This record does not assert independent reproduction. ### Avg entropy Brier Score: 0.391. Result ID: 69e9f93547f34ce924d1f451. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"VLA Model":"UniVLA"}. This record does not assert independent reproduction. ### Avg prob. Brier Score: 0.532. Result ID: 69e9f93547f34ce924d1f44f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"VLA Model":"UniVLA"}. This record does not assert independent reproduction. ### Running Avg prob. Brier Score: 0.544. Result ID: 69e9f93547f34ce924d1f450. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"VLA Model":"UniVLA"}. This record does not assert independent reproduction. ### Max prob. Brier Score: 0.899. Result ID: 69e9f93547f34ce924d1f44e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"VLA Model":"UniVLA"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.