RAP-WAM: Learning World-Action Models through Predictive Rehearsal and Action Perturbations

Key points

• Adapting these priors to control requires determining how future predictions should guide policy learning.1
• RAP-WAM combines predictive Rehearsal and Action Perturbations (RAP) with future-supervised queries and Coupled Action Projection for policy learning.1
• Predictive rehearsal trains on model-generated inputs while retaining recorded targets, and accumulates recurring errors into a spatial prior.1
• Action-response-based spatiotemporal credit assignment combines these responses, prediction error, and the rehearsal prior to allocate visual supervision; spatially aggregating the responses determines supervision intensity across future action steps.1

Sources (1)

• [1]RAP-WAM: Learning World-Action Models through Predictive Rehearsal and Action Perturbations
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 12, 09:09 AM

  Recent world action models draw on video generation priors to jointly predict future observations and robot actions.

  Adapting these priors to control requires determining how future predictions should guide policy learning.

Extractive summary: sentences quoted from the sources.