Failure Detection on ARC BC
14E-AURCATA
Evaluation results
| Method | Links | |
|---|---|---|
| ATAType=Ensembling heatmaps, threshold=.72026.03 | 14 | |
| ATAUncertainty Method=MCD, Aggregation parameter=.72026.03 | 13 | |
| PLMParameter=10, Uncertainty Framework=Maximum Softmax Probability (MSP)2026.03 | 12 | |
| PLMType=Ensembling heatmaps, value=102026.03 | 10 | |
| AVGType=Ensembling heatmaps2026.03 | 10 | |
| ICAType=Ensembling heatmaps2026.03 | 10 | |
| ICAUncertainty Framework=Maximum Softmax Probability (MSP)2026.03 | 10 | |
| PLMParameter=50, Uncertainty Framework=Maximum Softmax Probability (MSP)2026.03 | 10 | |
| AVGUncertainty Framework=Maximum Softmax Probability (MSP)2026.03 | 10 | |
| ATAThreshold=0.7, Uncertainty Framework=Maximum Softmax Probability (MSP)2026.03 | 10 | |
| ICAUncertainty Method=MCD2026.03 | 9 | |
| AVGUncertainty Method=MCD2026.03 | 9 | |
| AQAType=Ensembling heatmaps, threshold=.62026.03 | 8 | |
| PLMUncertainty Method=MCD, Aggregation parameter=502026.03 | 8 | |
| PLMParameter=20, Uncertainty Framework=Maximum Softmax Probability (MSP)2026.03 | 8 | |
| AQAThreshold=0.6, Uncertainty Framework=Maximum Softmax Probability (MSP)2026.03 | 8 | |
| AQAThreshold=0.9, Uncertainty Framework=Maximum Softmax Probability (MSP)2026.03 | 8 | |
| AQAThreshold=0.75, Uncertainty Framework=Maximum Softmax Probability (MSP)2026.03 | 8 | |
| PLMType=Ensembling heatmaps, value=202026.03 | 7 | |
| PLMType=Ensembling heatmaps, value=502026.03 | 7 | |
| AQAType=Ensembling heatmaps, threshold=.752026.03 | 7 | |
| AQAUncertainty Method=MCD, Aggregation parameter=.62026.03 | 7 | |
| GMM-IntUncertainty Framework=Maximum Softmax Probability (MSP)2026.03 | 7 | |
| GMM-IntType=Ensembling heatmaps2026.03 | 6 | |
| GMM-IntUncertainty Method=MCD2026.03 | 6 | |
| PLMUncertainty Method=MCD, Aggregation parameter=102026.03 | 6 | |
| PLMUncertainty Method=MCD, Aggregation parameter=202026.03 | 6 | |
| AQAUncertainty Method=MCD, Aggregation parameter=.752026.03 | 6 | |
| GMM-AllUncertainty Framework=Maximum Softmax Probability (MSP)2026.03 | 6 | |
| GMM-SpaType=Ensembling heatmaps2026.03 | 5 | |
| GMM-AllType=Ensembling heatmaps2026.03 | 5 | |
| BCAType=Ensembling heatmaps2026.03 | 5 | |
| AQAType=Ensembling heatmaps, threshold=.92026.03 | 5 | |
| ATAType=Ensembling heatmaps, threshold=.52026.03 | 5 | |
| GMM-AllUncertainty Method=MCD2026.03 | 5 | |
| GMM-SpaUncertainty Method=MCD2026.03 | 5 | |
| ATAUncertainty Method=MCD, Aggregation parameter=.32026.03 | 5 | |
| AQAUncertainty Method=MCD, Aggregation parameter=.92026.03 | 5 | |
| QFRUncertainty Framework=Maximum Softmax Probability (MSP)2026.03 | 5 | |
| BCAUncertainty Framework=Maximum Softmax Probability (MSP)2026.03 | 5 | |
| GMM-SpaUncertainty Framework=Maximum Softmax Probability (MSP)2026.03 | 5 | |
| QFRType=Ensembling heatmaps2026.03 | 4 | |
| ATAType=Ensembling heatmaps, threshold=.32026.03 | 4 | |
| QFRUncertainty Method=MCD2026.03 | 4 | |
| BCAUncertainty Method=MCD2026.03 | 4 | |
| ATAUncertainty Method=MCD, Aggregation parameter=.52026.03 | 4 | |
| ATAThreshold=0.5, Uncertainty Framework=Maximum Softmax Probability (MSP)2026.03 | 4 | |
| ATAThreshold=0.3, Uncertainty Framework=Maximum Softmax Probability (MSP)2026.03 | 4 |