ReAP: From Visual Failure Recognition to Action-Conditioned Risk Prediction for Runtime Failure Monitoring
To anticipate such risks before action execution, we introduce ReAP, a future-aware runtime monitoring framework that combines visual failure recognition with action-conditioned failure prediction.
ProofPaper ↗
Key points
- Despite rapid progress, generalist robot policies remain brittle during execution.
- An action-conditioned predictor then forecasts the resulting future representation in this space to estimate future failure risk.
- At runtime, the detector and predictor provide complementary signals for current-state failure recognition and future-aware risk assessment.
- Experiments across diverse robot policies in simulation and the real world demonstrate that ReAP effectively balances failure-detection accuracy and warning timeliness.
Sources (1)
- [1]ReAP: From Visual Failure Recognition to Action-Conditioned Risk Prediction for Runtime Failure MonitoringarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 12, 07:35 AM
To anticipate such risks before action execution, we introduce ReAP, a future-aware runtime monitoring framework that combines visual failure recognition with action-conditioned failure prediction.
Despite rapid progress, generalist robot policies remain brittle during execution.
Extractive summary: sentences quoted from the sources.