Learning Changes the World and Then What Was Learned has Changed: Training and Deploying Neural Networks in the Control Loop

Key points

• This paper introduces a unified taxonomy to categorize these sequential failure modes: the initial Closed-Loop Identification Error, the deployment-induced Action Error, and the continuous Tracking Error.1
• Through discrete-time sensitivity analysis, this work formalizes how offline-learned parameters locally guarantee a trajectory divergence upon deployment, degrading performance due to local loss geometry.1
• To mitigate this distribution shift, we propose actionable design policies centered on time-scale separated online adaptation.1
• These mechanisms are validated through ablation studies on a nonlinear Duffing oscillator, demonstrating that bounded continuous learning successfully mitigates trajectory divergence whereas static deployments fail, offering concrete guidelines for practitioners deploying machine learning components in safety-critical feedback systems.1

Sources (1)

• [1]Learning Changes the World and Then What Was Learned has Changed: Training and Deploying Neural Networks in the Control Loop
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 03:39 PM

  Neural networks offer critical adaptability for controlling complex systems where nominal modeling proves intractable.

  This paper introduces a unified taxonomy to categorize these sequential failure modes: the initial Closed-Loop Identification Error, the deployment-induced Action Error, and the continuous Tracking Error.

Extractive summary: sentences quoted from the sources.